Standards, Environments, and Macros qmail-spamthrottle(5) NNNNAAAAMMMMEEEE qmail-spamthrottle - the qmail spam throttle mechanism IIIINNNNTTTTRRRROOOODDDDUUUUCCCCTTTTIIIIOOOONNNN The idea of spam throttling came about after would-be spam- mers were easily circumventing (classic) tarpitting. A rea- sonable recipient limit in tarpitting must not adversely affect acceptable mail usage, so spam clients typically create multiple SMTP connections, all of which fall under this threshold. Other sources have similar concepts, using rate limiting, stuttering, et cetera to describe them. It was originally intended for use at ISPs to control their internal clients (users) SMTP usage, although it can applied equally in other environments. An ISP may wish to enable this mechanism for its customers to prevent them from using the mail servers as a convenient location from which to send spam. However, in some or all other cases (other originat- ing IP addresses) this mechanism might be disabled to allow for legitimate high-volume mail traffic such as mailing lists. Spam throttling acts in a similar manner to tarpitting, except that it is highly parameterized, more flexible, and (hopefully) more effective. A wait is imposed (via sssslllleeeeeeeepppp(3)) following the DDDDAAAATTTTAAAA command depending on these SMTP parameters: remote IP address; previous SMTP connection timestamp; and previous wait time. With the addition of teergrubing, spammers should keep their connections open and deliver less mail. DDDDEEEETTTTAAAAIIIILLLLSSSS Two files, _w_a_i_t and _t_i_m_e, store the previous wait time and SMTP connection timestamp, respectively. Both files are found in ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ssssppppaaaammmm////_d_i_r. Where _d_i_r is based on parame- ters set in ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttt. If _d_i_r is empty as a result, then it will be automatically set to _a/_b/0/0, where _a and _b are the two octets (in decimal) for the remote IP address, _a._b._c._d. Similarly, if _d_i_r starts with a slash (////), then it be automatically set to the _n-bit masked IP address (format [/_n]), based on the remote IP address. See qqqqmmmmaaaaiiiillll----ssssppppaaaammmmtttt((((5555)))) for details. NNNNooootttteeee:::: In case it is not yet evident, when _d_i_r is empty (or starts with a slash), as indicated above, then every dot (....) SunOS 5.11 Last change: 1 Standards, Environments, and Macros qmail-spamthrottle(5) is interpreted as a slash (////) in the construction of the directory where the spam throttle state files are stored. If you are using libtai for your time calculations, then the format for the _t_i_m_e file is a packed TAI64NA label. If you have perl and the tai64nlocal program, you can use the fol- lowing perl expression to convert from a packed TAI64NA label to a TAI64N timestamp: print join("","@",unpack("H24",<>)), "0; Given an entry in ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttt, such as ipblock:dir:st:stmax:flush:rcpt:tg:tg_resp: Message throughput is controlled via the value of _s_t. The delays imposed (by calling sssslllleeeeeeeepppp(3)) depend on: the value of _s_t); number of recipients for the current SMTP session (_R); the number of reasonable recipients per connection (_r_c_p_t); how much time has passed (_T) since the last SMTP request (as determined by ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ssssppppaaaammmm////_d_i_r////ttttiiiimmmmeeee); and the last imposed delay (_W) (as determined by ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ssssppppaaaammmm////_d_i_r////wwwwaaaaiiiitttt). The new delay is approximately (_R - _R / 2^(_R/_r_c_p_t)) * ((_W * _s_t * _R) / _T) when _r_c_p_t is greater than 0, and (_W * _s_t * _R) / _T otherwise. The unit of time is milliseconds. If _s_t_m_a_x is defined (and is non-zero), then it is used as a maximum (in milliseconds) for the delay calculated above. In short, _s_t is roughly the minimum time between messages and/or connections. If you already know that you only want a throughput of N messages per second, then you can use 1000/N as a good starting point for _s_t. CCCCOOOONNNNFFFFIIIIGGGGUUUURRRRAAAATTTTIIIIOOOONNNN For the following discussion, we assuming the matching entry in ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttt is ipblock:dir:st:stmax:flush:rcpt:tg:tg_resp: Despite efforts to impose a waiting period on would-be spam- mers, it is still possible for the client to circumvent the call to sssslllleeeeeeeepppp(3). That is, they may not wait for the SunOS 5.11 Last change: 2 Standards, Environments, and Macros qmail-spamthrottle(5) response from the DATA command, continuing to write their message, assuming success, then closing the socket, again without waiting for a response from the server; the message will be delivered at no (time) cost to them. Adherence to standards (such as ignoring the absence of PIPELINING) should not be assumed for clients acting as agents for unsolicited bulk email. As such, the _f_l_u_s_h variable can be set (non-zero) to indicate that all input will be flushed after calling sssslllleeeeeeeepppp(3) and prior to sending a response to the DATA command. RFC 2920 (STD 60) prohibits flushing of the input buffer if PIPELINING is supported. As such, EHLO responses will not advertise PIPELINING while _f_l_u_s_h is set. Another method, teergrubing, involves issuing continuation lines periodically to keep the client connected while they wait for the go ahead from the DATA command. By setting (non-zero) the variable _t_g, you can specify the frequency of continuation lines in response to the DATA command. If the argument to sssslllleeeeeeeepppp(3) would have been 11 (seconds) and _t_g is set to 2, then the response to the DATA command would result in several calls to sleep(2) (and one sleep(1)) with each accompanied by a continuation line. A continuation line consist of a 3-digit code, a dash, and an arbitrary string. The default string is "please wait", but can be changed using the _t_g__r_e_s_p variable. For example, ... DATA 354-please wait 354-please wait 354 go ahead ... EEEENNNNVVVVIIIIRRRROOOONNNNMMMMEEEENNNNTTTT The environment variable, TTTTCCCCPPPPRRRREEEEMMMMOOOOTTTTEEEEIIIIPPPP, is strictly required by spam throttle. If you are not using ttttccccppppsssseeeerrrrvvvveeeerrrr, then you will have to use ttttccccpppp----eeeennnnvvvv to ensure TTTTCCCCPPPPRRRREEEEMMMMOOOOTTTTEEEEIIIIPPPP is set. CCCCAAAAVVVVEEEEAAAATTTTSSSS The implicit translation of an empty directory to one based on the remote IP address will most certainly result in an unwieldy spam directory structure and should be reserved for small networks, such as the internal network side of an office or ISP (including ISP users). It is recommended that the /_n format be used in the default ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttt entry (empty network block). Then, for specific networks, a directory per IP address is still possible: for example, the entries SunOS 5.11 Last change: 3 Standards, Environments, and Macros qmail-spamthrottle(5) 192.168.0.0/24:/32::::::: :/16:1500:120000:::::: define the default spam throttle directory (assuming the remote IP address is _a._b._c._d) as _a/_b/0/0. However, when the remote IP address is in the 192.168.0.0/24 network block, the spam throttle directory will be _a/_b/_c/_d, since the _d_i_r parameter is ////33332222. EEEEXXXXAAAAMMMMPPPPLLLLEEEESSSS These examples assume that ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttthhhhrrrroooottttttttlllleeee contains a non-zero value. Here is a sample ////vvvvaaaarrrr////qqqqmmmmaaaaiiiillll////ccccoooonnnnttttrrrroooollll////ssssppppaaaammmmtttt file for a home user: # network:dir:st:stmax:flush:rcpt:tg:tg_resp: # # default entry (make it all share the public directory) :public:1500:120000:::::: # # private (trusted) network does not enforce spamthrot- tle 192.168.0.0/24::0:::::: # # some external network which we would like to throttle collectively 10.0.0.0/24:collected::::::: # # an external network (semi-trusted) which is throttled # based on individual IP address # - we don't specify SPAMTHROTTLEDIR and the default # behaviour of storing state files in directories # based on IP address is used) # - we also allow relaying from this semi-trusted # network 10.1.0.0/16:/32::::::: . Here is a sample file for a high-volume mail server (or servers) for some arbitrary ISP (with customer network 10.0.0.0/16 and internal/ employee network 10.1.0.0/24): # network:dir:st:stmax:flush:rcpt:tg:tg_resp: # # by default, turn throttling off ::0::::::: # # customer network uses default behaviour # (IP-based throttle files) SunOS 5.11 Last change: 4 Standards, Environments, and Macros qmail-spamthrottle(5) 10.0.0.0/16:/32::::::: # # employee network doesn't adhere to throttling 10.1.0.0/24::0:::::: # # external trusted network which legitimately # provides high volume mail traffic 10.1.1.0/24::0:::::: # # a collection of addresses/networks which we # might have gathered from past abuse experience # - we allow the mail, but we're aggressive # about throttling it 10.1.2.1/32:abuse:5000:::::: 10.1.2.2/32:abuse:5000:::::: 10.1.2.3/32:abuse:5000:::::: 10.1.3.0/24:abuse:5000:::::: . SSSSEEEEEEEE AAAALLLLSSSSOOOO tcp-env(1), tcp-environ(5), qmail-spamt(5), qmail-smtpd(8) AAAAUUUUTTTTHHHHOOOORRRR Dale Woolridge, James Law, and Moto Kawasaki. Contact the authors via email: . SunOS 5.11 Last change: 5