Dovecot ManageSieve Server
==========================

 1. Dovecot ManageSieve Server
     1. Introduction
     2. Versions and Downloading
         1. Tarball releases
         2. Mercurial repositories
     3. Patching Dovecot
         1. v1.0
         2. v1.1/v1.2
         3. Mercurial
     4. Compiling
         1. v1.0
         2. v1.1/v1.2
     5. Configuring
         1. Proxy
     6. Troubleshooting
         1. Manual Login and Script Upload
         2. Manual TLS Login
         3. Client Problems
     7. Known Issues
     8. Contact Info

Introduction
------------

The <Sieve plugin> [LDA.Sieve.txt] for Dovecot's /*<deliver> [LDA.txt]*/ LDA
expects a user's Sieve script to reside somewhere in the user's directory
(~/.dovecot.sieve by default). If the user is to be able to change his sieve
script, he needs shell or FTP access to his home directory, which is not always
desirable. This is especially applicable to mail servers with virtual users. As
a solution, theManageSieve protocol was proposed to manage sieve scripts on the
server without the need for direct file system access by the users.
Additionally, the Sieve scripts are compiled before they are installed, making
sure that the uploaded script is valid. This prevents a user from inadvertently
installing a broken Sieve script. The protocol specification
[http://tools.ietf.org/id/draft-martin-managesieve-10.txt] still has a draft
status, but it is already supported by quite a few (web)mail clients. Dovecot
now supportsManageSieve by means of an external patch and package.
Alternatively, an external python-based daemon called pysieved
[http://woozle.org/~neale/src/pysieved/] written by Neale Pickett can be used,
but this page only describes the native Dovecot implementation.

Versions and Downloading
------------------------

The ManageSieve daemon is available for Dovecot versions past and including
1.0. There is an important architectural difference between v1.0 and the later
implementations however. The v1.0 version is a very large patch that includes
another copy of the CMU Sieve library as used by deliver's<Sieve plugin>
[LDA.Sieve.txt]. In contrast, the v1.1 and later versions are largely
implemented as an external package with a small patch to enableManageSieve
service support in Dovecot itself. This means that these newer implementations
no longer include another copy of the CMU Sieve library: these use the Sieve
plugin package for compilation against the Sieve library. With the advent of
Dovecot v2.0, the patch that is now still necessary for Dovecot v1.X is likely
to disappear completely, as Dovecot will then include support for custom
services.

Tarball releases
----------------

The latest released versions of the ManageSieve implementation for Dovecot can
be downloaded from the following locations:

 * Dovecot v1.0: http://www.rename-it.nl/dovecot/1.0/
    * Patch filenames look as follows:
      *'dovecot-1.0.'*'X'*'-MANAGESIEVE-v'*'<version>'*'.diff.gz'*
 * Dovecot v1.1: http://www.rename-it.nl/dovecot/1.1/
    * Patch filenames look as follows:
      *'dovecot-1.1.'*'X'*'-managesieve-'*'<version>'*'.diff.gz'*

      Package filenames look as follows:
      *'dovecot-1.1-managesieve-'*'<version>'*'.tar.gz'*
 * Dovecot v1.2: http://www.rename-it.nl/dovecot/1.2/
    * Patch filenames look as follows:
      *'dovecot-1.2.'*'X'*'-managesieve-'*'<version>'*'.diff.gz'*

      Package filenames look as follows:
      *'dovecot-1.2-managesieve-'*'<version>'*'.tar.gz'*

The tarball releases are signed with public key 0x3DFBB4F4 which can be found
at wwwkeys.pgp.net.

Mercurial repositories
----------------------

The ManageSieve implementation is maintained in a set of Mercurial
repositories, just like Dovecot itself. If you need to use the very latest
version, even before it is released as a tarball, you can download it through
Mercurial. TheManageSieve repositories for the various Dovecot versions are
publicly available at:

---%<-------------------------------------------------------------------------
http://hg.rename-it.nl/
---%<-------------------------------------------------------------------------

For each Dovecot version, there is a dovecot-1.X-managesieve and a
dovecot-1.X-managesieve-patch repository. The -patch repositories contain the
patches against the various dovecot versions. These are so-called Mercurial
Queues (MQ) repositories. This provides an easy (quilt-like) means to manage
patches over an upstream source repository like the one that exists for
Dovecot. The patches are managed in a repository called 'patches' below the .hg
directory in your main repository. Currently, these MQ repositories only have
the managesieve.patch, which is the actual patch that is refreshed and released
each time Timo makes a new Dovecot release. The dovecot-1.x-managesieve
repositories contain the actualManageSieve implementation. As explained before,
this is missing for v1.0.

Using these repositories is explained in the next two sections that deal with
patching the Dovecot tree and compiling Dovecot andManageSieve.

Patching Dovecot
----------------

Applying the patch to Dovecot varies a little between the different versions.
Downloading from Mercurial is a special case.

v1.0
----

As shown above, the ManageSieve implementation for Dovecot v1.0 consists solely
of a large patch. You need to apply the downloaded '.diff.gz' patch to your
'dovecot-1.0' source tree. This is achieved by executing the following command
line inside the source tree (''../patchfile.diff.gz'' must be substituted with
the location of the patch file you downloaded):

---%<-------------------------------------------------------------------------
gzip -dc ../pathfile.diff.gz | patch -p1
---%<-------------------------------------------------------------------------

Compiling the patched sources is described in the next section.

v1.1/v1.2
---------

The ManageSieve implementations for v1.1 and v1.2 consist of a patch in
'.diff.gz' format and a separate '.tar.gz' package. You first need to patch and
compile the Dovecot sources. Applying the patch is achieved by executing the
following command line inside the dovecot source tree (''../patchfile.diff.gz''
must be substituted with the location of the patch file you downloaded):

---%<-------------------------------------------------------------------------
gzip -dc ../pathfile.diff.gz | patch -p1
---%<-------------------------------------------------------------------------

Once patched, you can compile Dovecot using the usual <build process>
[CompilingSource.txt]. So, unlike the v1.0 implementation, you do not need
'autotools'.

Mercurial
---------

The following command sequence shows how to incorporate the ManageSieve patch
repository in a freshly cloned dovecot-1.1 tree:

---%<-------------------------------------------------------------------------
hg clone http://hg.dovecot.org/dovecot-1.1
cd dovecot-1.1
hg clone http://hg.rename-it.nl/dovecot-1.1-managesieve-patch .hg/patches
hg qpush
---%<-------------------------------------------------------------------------

The procedure is identical for the other dovecot versions.

When updating the dovecot tree it is important to unapply all patches before
performing ''hg pull'' on the dovecot tree:

---%<-------------------------------------------------------------------------
hg qpop --all
hg pull
hg update
hg qpush
---%<-------------------------------------------------------------------------

This otherwise results in a multi-head situation which can be resolved using a
''hg rollback'' or using ''hg qpop --all'' afterwards, which should gracefully
clean up the situation (never merge the heads!).

Updating the patch repository works as follows:

---%<-------------------------------------------------------------------------
hg qpop --all
hg -R .hg/patches pull
hg -R .hg/patches update
hg qpush
---%<-------------------------------------------------------------------------

After 'hg qpush', the dovecot repository is patched and can be compiled as
explained in the next section (don't forget ./autogen.sh for compiling dovecot
in a mercurial repository).

Compiling
---------

v1.0
----

After applying the patch to the v1.0 Dovecot tree, the usual './configure',
'make', 'make install' sequence is not enough. First the 'automake'/'autoconf'
structure needs to be rebuilt to include theManageSieve sources in the
compilation process. This requires 'autotools' to be installed on your system.
When you downloaded using Mercurial, you have a script called 'autogen.sh' in
your source tree and you should proceed as specified<here>
[CompilingSource.txt]. Otherwise, execute the following command inside the
dovecot source tree:

---%<-------------------------------------------------------------------------
autoreconf -i
---%<-------------------------------------------------------------------------

Afterwards, you can continue the usual <build process> [CompilingSource.txt].

v1.1/v1.2
---------

The first prerequisite for compiling the ManageSieve service is a patched and
compiled dovecot tree, as explained in the previous section. Note that the
service will compile against an unpatched dovecot tree, but keep in mind that
Dovecot will not know about the existence ofManageSieve without the patch. The
second prerequisite is a compiled dovecot-sieve source tree. Compiling the
Sieve plugin is described<here> [LDA.Sieve.txt].

Now that you have both a compiled dovecot and a compiled dovecot-sieve source
tree, you can continue building theManageSieve service. You can either unpack
the '.tar.gz' package somewhere or clone the Mercurial repository. When you
have cloned the repository, you must first execute ''./autogen.sh'' inside the
source tree. For both alternatives the following commands subsequently need to
be executed inside the source tree:

---%<-------------------------------------------------------------------------
./configure --with-dovecot=<dovecot source tree>
--with-dovecot-sieve=<dovecot-sieve source tree>
make
sudo make install
---%<-------------------------------------------------------------------------

The parameters to './configure' represent the following:

 * --with-dovecot=<path>
    * Path to the compiled dovecot tree
 * --with-dovecot-sieve=<path>
    * Path to the compiled dovecot-sieve tree

Configuring
-----------

*NOTE*: If you have used the sieve plugin before and you have '.dovecot.sieve'
files in user directories, you are advised to *make a backup first*. Although
the managesieve daemon takes care to move these files to the sieve storage
before it is substituted with a symbolic link, this is not a very well tested
operation, meaning that there is a possibility that existing sieve scripts get
lost.

*NOTE*: This section describes the configuration for Dovecot v1.0 and v1.1. The
newly released v1.2 version differs in terms of configuration and you are
referred to its README
[http://hg.rename-it.nl/dovecot-1.2-managesieve/file/tip/README] file for a
proper explanation.

Along with all other binaries that dovecot uses, the managesieve and
managesieve-login binaries are installed during 'make install'. The only thing
you need to do to activate theManageSieve support in dovecot is to add
'managesieve' to the 'protocols=' configuration line in your dovecot.conf. The
managesieve daemon will listen on port 2000 by default. As the implementation
of the managesieve daemon is largely based on the original IMAP implementation,
it is very similar in terms of configuration. In addition to most<mail daemon
config settings> [MainConfig.txt], the managesieve daemon accepts a few more.
The following settings can be configured in the 'protocol managesieve' section:

listen = *:2000:
  IP or host address where to listen in for connections.

login_executable = /usr/libexec/dovecot/managesieve-login:
  Login executable location.

mail_executable = /usr/libexec/dovecot/managesieve:
  managesieve executable location. See mail_executable for IMAP for examples
  how this could be changed.

managesieve_max_line_length = 65536:
  Maximum managesieve command line length in bytes. This setting is directly
  borrowed from IMAP. But, since long command lines are very unlikely
  withManageSieve, changing this will not be very useful.

sieve_storage = :
  This specifies the path to the directory where the uploaded scripts are
  stored. In terms of '%' variable substitution it is identical to dovecot's
  'mail_location' setting used by the mail protocol daemons. Scripts are stored
  as separate files with extension '.sieve', all other files are ignored when
  scripts are listed by aManageSieve client. If this setting remains
  unspecified, the 'mail_location' setting is used as explained below.

sieve = ~/.dovecot.sieve:
  Specifies the location of the symbolic link pointing to the active script in
  the sieve storage directory. This must match the<sieve setting used by
  deliver> [LDA.Sieve.txt]. Variable substitution with % is recognized. If a
  regular file exists at this location, it is moved to the 'sieve_storage'
  location before the symbolic link is installed. It is renamed to
  'dovecot.orig.sieve' and therefore listed as 'dovecot.orig' by a ManageSieve
  client.

mail_location =:
  If, for some inobvious reason, the sieve_storage remains unset, the
  managesieve daemon uses the specification of the mail_location to find out
  where to store the sieve files. However, this is provided only for backwards
  compatibility and you should always use the sieve_storage setting in stead.

managesieve_implementation_string = dovecot:
  To fool ManageSieve clients that are focused on CMU's timesieved you can
  specify the IMPLEMENTATION capability that the dovecot reports to clients
  (e.g. ''Cyrus timsieved v2.2.13'').

Scripts are stored at the location specified by the 'sieve_storage' setting.
The active sieve script is managed as a symbolic link pointing to the active
script in the sieve storage direcotory. The location of this symlink can be
specified with the 'sieve' setting. Make sure this setting is identical to what
<deliver> [LDA.txt] is using for the <Sieve plugin> [LDA.Sieve.txt]. The
default location is '~/.dovecot.sieve'. Note that if a file starting with '.'
is placed inside a Maildir, it will be recognized as a folder, so try to avoid
that.

The current version of the managesieve daemon places the script storage
directory in the mail folder as specified by the 'mail_location' setting if no
sieve_storage is specified. Actually, it is placed in the 'CONTROL=' directory
of 'mail_location' if specified, otherwise the 'sieve' directory is placed in
the root of the mail location. In a mail or mail control directory, scripts are
always stored in a 'sieve' subdirectory. Note that for some mail storage types
(e.g. mbox) this script directory is listed as a mail folder, so be sure to put
the sieve scripts somewhere else if you can.

A storage location specified by 'sieve_storage' is always generated
automatically if it does not exist (as far as the system permits the user to do
so; no root privileges are used). This is similar to the behaviour of the mail
daemons. Note that when 'mail_location' is used to specify the script storage
location, only the 'sieve' subdirectory is generated automatically.

The following provides an example configuration for ManageSieve in
dovecot.conf. Only sections relevant toManageSieve are shown. Refer to
dovecot-example.conf in your patched dovecot tree for a full example with
comments, but don't forget to add 'managesieve' to the 'protocols' setting if
you use it.

---%<-------------------------------------------------------------------------
# Start imap, pop3 and managesieve services
protocols = imap pop3 managesieve

protocol managesieve {
  # Specify an alternative address:port the daemon must listen on
  # (default: *:2000)
  #listen = localhost:2000

  sieve=~/.dovecot.sieve
  sieve_storage=~/sieve
}
---%<-------------------------------------------------------------------------

Proxy
-----

Like Dovecot's imapd, the ManageSieve login daemon supports proxying to
multiple backend servers. Although the underlying code is copied from the imapd
sources for the most part, it has someManageSieve-specifics that have not seen
much testing. The<proxy configuration wiki page>
[PasswordDatabase.ExtraFields.Proxy.txt] for POP3 and IMAP should apply to
ManageSieve as well.

Troubleshooting
---------------

Like Dovecot itself, *the ManageSieve service always logs a detailed error
message* if something goes wrong at the server (refer to < Dovecot Logging>
[Logging.txt] for more details): the logs are the first place to look if you
suspect something is wrong. To get additional debug messages in your log file,
you should set 'mail_debug=yes' in <dovecot.conf> [MainConfig.txt] (inside
'protocol managesieve {...}'if you want to enable this for managesieve only).

If the client commits protocol violations or sends invalid scripts, an error
response is provided to the client which is not necessarily logged on the
server. A goodManageSieve client presents such error messages to the user.

Manual Login and Script Upload
------------------------------

If you fail to login or upload scripts to the server, it is not necessarily
caused by Dovecot or your configuration. It is often best to test
yourManageSieve server manually first. This also provides you with the direct
error messages from the server without intermission of your client. If you do
not use TLS, you can connect using a simple 'telnet' or 'netcat' connection to
the configured port. Otherwise you must use a TLS-capable text protocol client
like 'gnutls-cli' as described below. Upon connection, the server presents the
initial greeting with its capabilities:

---%<-------------------------------------------------------------------------
"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify
include envelope body relational regex subaddress copy"
"STARTTLS"
OK "Dovecot ready."
---%<-------------------------------------------------------------------------

Note that the reported 'STARTTLS' capability means that the server accepts TLS,
but, since you are using telnet/netcat, you cannot use this (refer to Manual
TLS Login below). The 'SASL' capability lists the available SASL authentication
mechanisms. If this list is empty and 'STARTTLS' is available, it probably
means that the server forces you to initiate TLS first (as dictated by
''disable_plaintext_auth=yes'' in <dovecot.conf> [MainConfig.txt]).

Now you need to log in. Although potentially multiple SASL mechanisms are
available, only 'PLAIN' is described here. Authentication is performed using
theManageSieve 'AUTHENTICATE' command. This command typically looks as follows
when the 'PLAIN' mechanism is used:

---%<-------------------------------------------------------------------------
AUTHENTICATE "PLAIN" "<base64-encoded credentials>"
---%<-------------------------------------------------------------------------

The credentials are the base64-encoded version of the string
'"\0<username>\0<password"' (in which '\0' represents the ASCII NUL character).
Generating this is cumbersome and a bit daunting for the novice user, so for
convenience a simple Perl script is provided to generate the 'AUTHENTICATE'
command for you. It is available here
[http://www.rename-it.nl/dovecot/utilities/sieve-auth-command.pl] and used as
follows:

---%<-------------------------------------------------------------------------
sieve-auth-command.pl <username> <password>
---%<-------------------------------------------------------------------------

The command is written to stdout and you can paste this to your protocol
session, e.g.:

---%<-------------------------------------------------------------------------
AUTHENTICATE "PLAIN" "AHVzZXJuYW1lAHBhc3N3b3Jk"
OK "Logged in."
---%<-------------------------------------------------------------------------

Now that you are logged in, you can upload a script. This is done using the
'PUTSCRIPT' command. Its first argument is the name for the script and its
second argument is a string literal. A string literal starts with a length
specification ''{<bytes>+}'' followed by a newline. Thereafter the server
expects '<bytes>' bytes of script data. The following uploads a trivial 6 byte
long sieve script that keeps every message (6th byte is the newline character):

---%<-------------------------------------------------------------------------
PUTSCRIPT "hutsefluts" {6+}
keep;
OK "Putscript completed."
---%<-------------------------------------------------------------------------

Upon successful upload, you should find a file called 'hutsefluts.sieve' in
your 'sieve_storage' directory. The script should also be listed by the server
as follows when the 'LISTSCRIPTS' command is issued:

---%<-------------------------------------------------------------------------
LISTSCRIPTS
"hutsefluts"
OK "Listscripts completed."
---%<-------------------------------------------------------------------------

You can check whether your script is uploaded correctly by downloading it using
the 'GETSCRIPT' command. This command accepts the name of the downloaded script
as its only parameter:

---%<-------------------------------------------------------------------------
GETSCRIPT "hutsefluts"
{6}
keep;
OK "Getscript completed."
---%<-------------------------------------------------------------------------

To let the Sieve plugin use your newly uploaded script, you must activate it
using the 'SETACTIVE' command (only one script can be active at any time). The
active script is indicated 'ACTIVE' in the 'LISTSCRIPTS' output, e.g.:

---%<-------------------------------------------------------------------------
SETACTIVE "hutsefluts"
OK "Setactive completed."
LISTSCRIPTS
"hutsefluts" ACTIVE
OK "Listscripts completed.
---%<-------------------------------------------------------------------------

The symbolic link configured with the 'sieve' setting should now point to the
activated script in the 'sieve_storage' directory. If no script is active, this
symbolic link is absent.

Manual TLS Login
----------------

When TLS needs to be used during manual testing, 'gnutls-cli' provides the
means to do so. This command-line utility is part of the GNUTLS distribution
and on most systems this should be easy to install. It is used to connect
toManageSieve as follows:

---%<-------------------------------------------------------------------------
gnutls-cli --starttls -p <port> <host>
---%<-------------------------------------------------------------------------

This starts the client in plain text mode first. As shown in the previous
section, the server presents a greeting with all capabilities of the server. If
'STARTTLS' is listed, you can issue the 'STARTTLS' command as follows:

---%<-------------------------------------------------------------------------
STARTTLS
OK "Begin TLS negotiation now."
---%<-------------------------------------------------------------------------

If an OK response is given by the server you can press 'Ctrl-D' to make
'gnutls-cli' start the TLS negotiation. Upon pressing 'Ctrl-D', 'gnutls-cli'
will show information on the negotiated TLS session and finally the first
response of the server is shown:

---%<-------------------------------------------------------------------------
"IMPLEMENTATION" "dovecot"
"SASL" "PLAIN"
"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags notify
include envelope body relational regex subaddress copy"
OK "TLS negotiation successful."
---%<-------------------------------------------------------------------------

Hereafter, you can continue to authenticate and upload a script as described in
the previous section.

Client Problems
---------------

If manual efforts to upload a script are successful, but your client still
fails, you need to obtain a view on what the client communicates with the
server. A common method is to sniff the client protocol session using a tool
like 'ngrep'. However, this will not work when TLS is active. If the problem is
not specific to TLS, you are advised to temporarily turn off TLS and sniff the
plain text protocol. If TLS is part of the issue, you can use Dovecot's<
rawlog> [Debugging.Rawlog.txt] facility to see what is going on if the client
is logged in. If the authentication is the problem, there is no real nice way
to obtain a transcript of the protocol. One way is to run managesieve from
inetd, wrapping it into a script that writes the protocol messages somewhere
(*FIXME*: This needs some checking and explanation). Alternatively, if
possible, the client can be altered to write its protocol messages somewhere.

Known Issues
------------

 * Although this ManageSieve server should comply with the draft specification
   of theManageSieve protocol, quite a few clients don't. This is particularly
   true for the TLS support. However, now that Cyrus' Timsieved has changed its
   behaviour towards protocol compliance, all those clients will follow
   eventually.

   The core of the TLS problem is that a ManageSieve server is required to send
   an unsolicited CAPABILITY response right after successful TLS negotiation.
   Older Cyrus servers did not do this and many clients incorporated this
   protocol error as the standard, meaning that these do not expect the
   CAPABILITY response and thus fail with subsequent commands.
   The following clients are known to have this TLS issue:
   Thunderbird Sieve add-on:
     TLS broken for old versions. Starting with version 0.1.5 the Thunderbird
     Sieve add-on properly supports TLS.

   KMail + kio_sieve:
     TLS broken for old versions. This issue is fixed at least in kmail 1.9.9 /
     kde 3.5.9.

   SquirrelMail/AvelSieve:
     For some users the Avelsieve client stores scripts but fails to retrieve
     them later. This problem is directly caused by<AvelSieve.txt>'s TLS
     support. A quick way to fix this is not to enable TLS forManageSieve.
     <AvelSieve.txt> stable (v1.0.1) does not have TLS support at all, so you
     will see this happen only with the development or SVN versions.
     Another issue is that (at least with avelsieve-1.9.7) it is impossible to
     delete the last rule of a script.
     For avelsieve-1.9.7 you find a patch that fixes these two issues here
     [http://www.rename-it.nl/dovecot/client-patches/avelsieve-1.9.7-dovecot.diff].

 * Other client issues:
   SmartSieve, WebSieve:
     These clients are specifically written for Cyrus timsieved and fail on
     multiple stages of the protocol when connected to DovecotManageSieve.
      * /(Stephan Bosch)/: I intend to look at these in the future, but
        currently these are very much unavailable for use with Dovecot. But,
        feel free to fix these yourself:)

 * The current implementation of the daemon does not have quota enforcement as
   recommended in the specification. So keep in mind that malicious users could
   fill your file system with loads of spurious script files.
 * The ANONYMOUS authentication mechanism is currently not supported and
   explicitly denied.

*NOTE*: If you add new issues to this list, notify the author or send an e-mail
to the Dovecot mailing list as described below. In any case, you must make sure
that the issue is properly explained and that the author can contact you for
more information.

Contact Info
------------

 * Author: Stephan Bosch, stephan@rename-it.nl
 * IRC: Freenode, #dovecot, S[r]us
 * Please use the Dovecot mailing list
   [http://www.dovecot.org/mailinglists.html] for questions about the
   ManageSieve service. You don't have to subscribe to it. 

(This file was created from the wiki on 2009-01-05 04:42)