Services/Mail: Difference between revisions

From DcUsers
No edit summary
(31 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Template:Service
{{Template:Service
| description = Available services:
| description = Available services:
* multiple email addresses (mailboxes or redirections) based on ''milkypond.org'' (or maybe ''duckcorp.org'') domain(s) or you own domain(s)
* multiple email addresses (mailboxes+aliases or redirections) based on the ''milkypond.org'' (or maybe ''duckcorp.org'') domain
* roaming accounts to be able to send emails wherever you are
* roaming accounts to be able to send emails wherever you are
If you want email addresses using you own domain(s), see the [[Services/MailHosting|mail hosting service]].
| prerequisite = You need to ask an administrator for an account.
| prerequisite =
| account = global
| account = global-reg
| ipv6 = true
| ipv6 = true
| security_notes = Access to the mail services are fully secured, and to your data, nevertheless most mail exchange on the Internet are not. Complete security can be achieved using signed and encrypted mail (see [[wikipedia:Pretty_Good_Privacy|OpenPGP]] and [[wikipedia:S/MIME|S/MIME]])
| security_notes =
}}
}}


Line 21: Line 22:


You can use our servers to send mails out too via smtp.duckcorp.org using:
You can use our servers to send mails out too via smtp.duckcorp.org using:
* SMTP+TLS (TCP 25) / SMTPS (TCP 465)
* SUBMISSION (TCP 587) which is the recommended way and has less chance to be filtered, or alternatively SMTP+TLS (TCP 25) / SMTPS (TCP 465)
* and SASL authentication
* and SASL authentication


=== Using a Web Interface ===
=== Using a Web Interface ===


Several [https://webmail.duckcorp.org/ webmail interfaces] are available.
A [https://webmail.duckcorp.org/ webmail interface] is available.


== Mailboxes Features ==
== Antivirus and Antispam ==


Mails stored on our server are checked upon arrival for viruses and SPAMs. Outgoing mails from our services are checked too.
=== Antivirus and Antispam ===


Mails being viruses are suppressed automatically upon arrival. Mails with a huge probability of being SPAM are destroyed too. Good mails and possibly spammy mails are delivered to your mailbox.
Mails stored on our server are checked upon arrival for viruses and SPAMs. Mail redirected to a third party mailbox are not, not to mess with the strategy of the the final destination (which could arm learning greatly).


SPAMs are more difficult to detect, and it is an error-prone process, so we chose to use a learning software instead of using a global database. After a few weeks of teaching it is able to recognize most of your habits and block a lot of annoying SPAM.
Mails being viruses are suppressed automatically upon arrival. Once in a week, an extra antivirus check is done on all hosted mailboxes, to remove viruses not detected at the time it arrived (our antivirus database is updated daily), so you may see such mail disappear from your mailbox, don't fear.


=== Probable SPAM Notification ===
SPAMs are more difficult to detect, and it is an error-prone process, so we chose to use a learning software intead of using a global database. On the bright side of things, you would never loose mails and the software should be able to learn what is a SPAM from '''your''' point of view (which has proved to differ slightly between our users), but on the grey side you will have to teach him regulary to avoid errors. When your account is created, it acts like an empty headed body letting everything through, but after a few weeks of teaching it is able to recognise most of your habits and block a lot of annoying SPAM.


Mails are delivered normally with a special field added (''X-Spam-Status'') to your mail headers (not always visible depending on your mail client and its configuration) indicating if the mail is a SPAM and its ''spamminess'' score.
To teach the system, you can:
* use the [https://spamfilter.duckcorp.org/ web interface] to retrain from errors in the ''History'' page
* resend SPAM to dc-spam@duckcorp.org and HAM (non-SPAM) to dc-ham@duckcorp.org
*: {{warning}} PLEASE TAKE CARE to use the ''resend'' function of your mailer and never use ''forward'', which would result in '''yourself''' being considered as spam or ham


This is quite handy if you prefer SPAMs arranged in a specific folder. You may use then the provided [[#Mail Filtering |filtering system]] to sort them properly, or configure your eMail client.
You can yourself setup a few parameters in the ''Preferences'' page of the [https://spamfilter.duckcorp.org/ web interface], mostly:
* how the system should train:
** TEFT (''On every new message scanned by the filter''): useful to learn very quickly especially on a new mailbox or when you've got tons of spam on your addresses, but you need to train it regulary (daily is better) or it will learn wrong (very quickly too !)
** TOE (''Only when the filter makes a mistake''): slow learning, but if you don't have time for training very often or go on long holidays without internet access this mode will give you a stable behavior
** TUM (''Only with new data or if the filter makes a mistake''): trade-off between the two previous modes, learn quite fast and not too demanding, adapts better to new SPAMs then TOE
* filter sensitivity: you can ask the system to be more or less aggressive
* message handling:
** ''Quarantine the message'': mails are not delivered to your mailbox but retained by the system and you can only manage them through the [https://spamfilter.duckcorp.org/ web interface] ; if you retrain a quarantined mail as HAM, it will be freed from quarantine and delivered in your mailbox at once ; in this mode, you need to flush your quarantine once in a while
** ''Tag the Subject header with'': mails are delivered with a modified title (which is a tad ugly)
** ''Deliver the message normally with a X-DSPAM-Result header'': mails are delivered with a special field in your mail headers (not always visible depending on your mail client and its configuration) ; this is quite handy if you prefer SPAMs arranged in a specific folder
* ''Disable DSPAM filtering'': you may switch off filtering at your own risk
We strongly recommend using TEFT with daily training for a few weeks then switching to TUM. If you need to go away for a long time, switching to TOE is recommended. Using the quarantine feature is also a safe choice, as the web interface is quite easy to use.


=== Mail Filtering ===
=== Retraining ===


Teaching the system what is SPAM and HAM (non-SPAM) is called ''retraining''.
With your favourite mail client, you can probably filter your mails in proper folders already. Nevertheless, this can be a annoying operation:

If you have of create a folder named ''Junk'', then it is automatically magic:
* if you drop mail into it, it is automatically retrained as SPAM
* if you move it out of it, it is automatically retrained as HAM
* with time, certain old mails automatically expire, see below

To use this folder properly, SPAMs '''must''' be delivered in the ''Junk'' folder automatically, so you can move them out in case of error.
You just need to subscribe to this folder in your mail software (this folder is always auto-created). It is not recommended to use your mail software for this filtering, this is horribly inefficient and you would have to set things up on each device and manually synchronize your settings.

Automatic expiration logic, based on delivery time (and not the mail timestamp):
* deleted mails are purged after 1 day
* read mails without an important flag are purge after 7 days
* unread mails without an important flag are purge after 30 days
If you use the ''incoming_spam'' [[#Global_Rules|global rule]], then SPAMs newly discovered by the system are flagged as important, which means these mails will never by purged until you review it and decide to delete them or mark them as not important. Custom rules might play with this flag and achieve an automatic selection.

== Mail Filtering ==

With your favourite mail client, you can probably filter your mails in proper folders already. Nevertheless, this can be an annoying operation:
* blocking you mail client for a long time if you have to process plenty of mails
* blocking you mail client for a long time if you have to process plenty of mails
* downloading each mail information, and sometimes content (depending on your filters), is lenghty too, and cost much bandwidth
* downloading each mail information, and sometimes content (depending on your filters), is lenghty too, and cost much bandwidth
* syncing filters accross your machines (home desktop, laptop, office machine…) is a pain in the ass
* syncing filters across your machines (home desktop, laptop, office machine…) is a pain in the ass
* processing only when you're online prevents triggering actions in a timely manner (automatic redirect, vacation messages…), and running a machine 24/7 with a mail client polling new mails every 30s is not a solution
* processing only when you're online prevents triggering actions in a timely manner (automatic redirect, vacation messages…), and running a machine 24/7 with a mail client polling new mails every 30s is not a solution


We provide a much better way to do this using the [[wikipedia:Sieve_(mail_filtering_language)|SIEVE]] filters. Shortly, SIEVE is a language dedicated to expressing mail filters. Our server is able to process your mails according to these filters as soon as they arrive. You then don't have to care about them anymore, and may use light mail clients or webmails when you're not on your machine with your favourite software. To push your filters on the server, a dedicated protocol exists:
We provide a much better way to do this using the [[wikipedia:Sieve_(mail_filtering_language)|SIEVE]] filters. Shortly, SIEVE is a language dedicated to expressing mail filters (also called ''rules''). Our server is able to process your mails according to these filters as soon as they arrive. You then don't have to care about them anymore, and may use light mail clients or webmails when you're not on your machine with your favourite software.
: MANAGESIEVE (TCP 4190) on sieve.duckcorp.org


=== Rules Configuration ===
Several softwares support managing SIEVE rules:

* Icedove/Thunderbird using the SIEVE extension (in xul-ext-sieve Debian package) provides a rules editor
You can express sort/reject/vacation/… filters using these rules, as the capabilities are very rich. Several softwares support managing SIEVE rules:
* Horde (Ingo) provides an easy to use web interface
* Icedove/Thunderbird:
* Squirrelmail with the Avelsieve extension provides an easy to use web interface
*: using the SIEVE extension (in ''xul-ext-sieve'' Debian package), it provides a rules editor (for power-users)
* Roundcube with an extension provides an easy to use web interface
* Roundcube:
* sieve-connect provides a CLI to upload/download/activate your rules files
*: coupled with the sieverules extension, provides an easy to use web interface
*: this webmail has been made available [https://webmail.duckcorp.org/ here]
* sieve-connect:
*: provides a CLI to upload/download/activate your rules files
*: this tool is available on [[Services/Shell|shell hosts]]
(tell us if you know more software supporting this feature)
(tell us if you know more software supporting this feature)

To push your filters on the server, a dedicated protocol exists: MANAGESIEVE (TCP 4190) on ''sieve.duckcorp.org''. Our webmails are already configured to use it, but it you use sieve-connect from our hosts or your own mail software, you'll need these parameters.


You can read more info about SIEVE here:
You can read more info about SIEVE here:
Line 80: Line 91:
* http://sieve.info/
* http://sieve.info/


=== Global Rules ===
=== Fetching eMails from an External Mailbox ===

Global rules are provided to ease configuration on specific filters. They can easily be included in your own configuration.

Available rules:
* ''incoming_spam'':
*: SPAMs will automatically be delivered in the ''Junk'' folder and marked as ''important''

If you write your own custom rules, here is an example on how to use one of them:
require ["include"];
include :global "<rule-name>";

=== Default Rules ===

The default settings (since 2112-02-11) are to use the global ''incoming_spam'' rule (see below), in order to provide a simple default configuration for most users. All other mails will end-up in your ''Inbox'' folder and you may then sort them by yourself.

If you create your own rules, the default rules won't apply anymore, so power-users can replace the default behavior completely to achieve what they really need. If you want to use the default behavior you can use:
require ["include"];
include :global "incoming_spam";

You can also take advantage of the antispam spaminess score to use a different threshold like this:
require ["fileinto", "spamtestplus", "relational", "comparator-i;ascii-numeric"];
# if SPAM score is >37% then move into the junk box
if spamtest :percent :value "gt" :comparator "i;ascii-numeric" "37" {
fileinto "Junk";
stop;
}

== Fetching eMails from an External Mailbox ==


{{todo}}
{{todo}}


=== Shared/Public Folders ===
== Shared/Public Folders ==


You may need to share mails or messages with friends or people you do stuff with (in a project or association). Depending on your needs, two solutions are possible:
You may need to share mails or messages with friends or people you do stuff with (in a project or association). Depending on your needs, two solutions are possible:
* share some of your own private folders, thus called ''shared folders''
* share some of your own private folders, thus called ''shared folders''
* manage a special folder hierarchy, called ''public folders'' (even if they may not be accessible to everyone)
* manage a special folder hierarchy, called ''public folders'' (even if they may not be accessible to everyone)
*: if you need one, ask an administrator; the namespace name can be freely chosen but must be unambiguous and is subject to approval


=== Namespaces ===
Through IMAP, or our webmails internally using IMAP, it is possible to partition the folder hierarchy into ''namespaces''. Traditionally you are using the root namespace for your private folders. Additionnal namespaces can be created and will appear among your own folders or separate, depending on your mail client's choice of representation. To avoid name clashes, we decided to prefix all additionnal namespace names with a '''#'''.

Through IMAP, or our webmails internally using IMAP, it is possible to partition the folder hierarchy into ''namespaces''. Traditionally you are using the root namespace for your private folders. Additional namespaces can be created and will appear among your own folders or separate, depending on your mail client's choice of representation. To avoid name clashes, we decided to prefix all additional namespace names with a '''#'''.


Since 2011-05-14, the following extra namespaces are created and reserved:
Since 2011-05-14, the following extra namespaces are created and reserved:
* #Shared, containing all folders other users decided to share with you
* #Shared, containing all folders other users decided to share with you
* #MilkyPond, containing public MP/DC information mailboxes you may subscribe at will
* #MilkyPond, containing public MP/DC informational mailboxes you may subscribe at will


=== Folders Permissions ===
Using IMAP, it is possible to setup rights (read only, write allowed…) to your own folders in order to share them with other users, or group of users. Public folders are owned by noone, and must be created by DC administrators ; if you need one, send us a request and we may be able to create it and delegate administration to your care (namespace name is subject to approval). Once created, public folders can be administered like shared folders.

Using IMAP, it is possible to setup rights (read only, write allowed…) to your own folders in order to share them with other users, or group of users.

Public folders are owned by no-one, and must be created by the administrators. Once your request is accepted we will delegate its administration to your care. It can then can be managed like shared folders.

=== Software Support ===


Client mail softwares support:
Client mail softwares support:
* RoundCube:
* Horde: support namespaces, shared folders configurable via ''Options->Share Folders'' menu but no ACLs for public folders
* Icedove/Thunderbird: support namespaces, shared and public folders configurable via folder selection and ''Tools->Imap-ACL'' menu action
*: support namespaces, shared and public folders configurable via ''Settings->Folders'', selecting a folder then using the ''Sharing'' tab
* Icedove/Thunderbird:
*: support namespaces, shared and public folders configurable via folder selection and ''Tools->Imap-ACL'' menu action

Most other softwares have namespace support only, so you should be able to use shared/public folders you have rights on but not configure them yourself
Most other softwares have namespace support only, so you should be able to use shared/public folders you have rights on but not configure them yourself
(tell us if you know more software supporting this feature)
(tell us if you know more software supporting this feature).


== Limitations ==
== Limitations ==
Line 108: Line 162:
=== Maximum Mail Size ===
=== Maximum Mail Size ===


Mail you send or receive are limited to 10MB. If you need to transmit much bigger data, then a mail transport is not appropriate, you'd better use a file sharing method instead.
Mail you send or receive are limited to '''20MB'''. If you need to transmit much bigger data, then a mail transport is not appropriate, you'd better use a file sharing method instead.


=== Quotas ===
=== Quotas ===
Line 114: Line 168:
Even if it would be nice to live without it, we had to establish quotas to force people sort their mails out once in a while and delete useless things instead of leaving an ever-growing mess behind.
Even if it would be nice to live without it, we had to establish quotas to force people sort their mails out once in a while and delete useless things instead of leaving an ever-growing mess behind.


The default quota is 512MB which is not that big but should match needs of most users. This said, you may ask us for more and there's no reason we would refuse a reasonable demand.
The default quota is '''512MB''' which is not that big but should match needs of most users. This said, you may ask us for more and there's no reason we would refuse a reasonable demand.


== Technical Details ==
== Technical Details ==


This service is made using:
This service is made using:
* [http://www.postfix.org/ Postfix]
* [https://www.postfix.org/ Postfix]
* [http://dovecot.org/ Dovecot]
* [https://dovecot.org/ Dovecot]
* [https://projects.duckcorp.org/projects/spoolinger Spoolinger]
* [http://dspam.sourceforge.net DSPAM]
* [https://rspamd.com/ Rspamd]
* [http://johannes.sipsolutions.net/Projects/dovecot-antispam dovecot-antispam]
* [http://www.horde.org/ Horde]
* [https://www.roundcube.net/ RoundCube]
* [http://www.squirrelmail.org/ SquirrelMail]
* [http://www.roundcube.net/ RoundCube]

Revision as of 08:41, 19 December 2018

Service 'Mail'
Description Available services:
  • multiple email addresses (mailboxes+aliases or redirections) based on the milkypond.org (or maybe duckcorp.org) domain
  • roaming accounts to be able to send emails wherever you are

If you want email addresses using you own domain(s), see the mail hosting service.

Prerequisite None
Account Global (registration required) IPv6 Ready Yes
Security Notes Access to the mail services are fully secured, and to your data, nevertheless most mail exchange on the Internet are not. Complete security can be achieved using signed and encrypted mail (see OpenPGP and S/MIME)


Access

Using a Mail Reader

You can retrieve your mails, in case of a real mailbox, using either:

  • IMAP+TLS (TCP 143) / IMAPS (TCP 993) on imap.duckcorp.org
  • or POP3+TLS (TCP 110) / POP3S (TCP 995) on pop.duckcorp.org

The IMAP protocol is recommended over POP3, as it provides many interesting features. If you want to download all your mails absolutely at home, loosing the ability to read your mail from anywhere on the planet, you can do that with IMAP too (look at your mail client settings).

You can use our servers to send mails out too via smtp.duckcorp.org using:

  • SUBMISSION (TCP 587) which is the recommended way and has less chance to be filtered, or alternatively SMTP+TLS (TCP 25) / SMTPS (TCP 465)
  • and SASL authentication

Using a Web Interface

A webmail interface is available.

Antivirus and Antispam

Mails stored on our server are checked upon arrival for viruses and SPAMs. Outgoing mails from our services are checked too.

Mails being viruses are suppressed automatically upon arrival. Mails with a huge probability of being SPAM are destroyed too. Good mails and possibly spammy mails are delivered to your mailbox.

SPAMs are more difficult to detect, and it is an error-prone process, so we chose to use a learning software instead of using a global database. After a few weeks of teaching it is able to recognize most of your habits and block a lot of annoying SPAM.

Probable SPAM Notification

Mails are delivered normally with a special field added (X-Spam-Status) to your mail headers (not always visible depending on your mail client and its configuration) indicating if the mail is a SPAM and its spamminess score.

This is quite handy if you prefer SPAMs arranged in a specific folder. You may use then the provided filtering system to sort them properly, or configure your eMail client.

Retraining

Teaching the system what is SPAM and HAM (non-SPAM) is called retraining.

If you have of create a folder named Junk, then it is automatically magic:

  • if you drop mail into it, it is automatically retrained as SPAM
  • if you move it out of it, it is automatically retrained as HAM
  • with time, certain old mails automatically expire, see below

To use this folder properly, SPAMs must be delivered in the Junk folder automatically, so you can move them out in case of error. You just need to subscribe to this folder in your mail software (this folder is always auto-created). It is not recommended to use your mail software for this filtering, this is horribly inefficient and you would have to set things up on each device and manually synchronize your settings.

Automatic expiration logic, based on delivery time (and not the mail timestamp):

  • deleted mails are purged after 1 day
  • read mails without an important flag are purge after 7 days
  • unread mails without an important flag are purge after 30 days

If you use the incoming_spam global rule, then SPAMs newly discovered by the system are flagged as important, which means these mails will never by purged until you review it and decide to delete them or mark them as not important. Custom rules might play with this flag and achieve an automatic selection.

Mail Filtering

With your favourite mail client, you can probably filter your mails in proper folders already. Nevertheless, this can be an annoying operation:

  • blocking you mail client for a long time if you have to process plenty of mails
  • downloading each mail information, and sometimes content (depending on your filters), is lenghty too, and cost much bandwidth
  • syncing filters across your machines (home desktop, laptop, office machine…) is a pain in the ass
  • processing only when you're online prevents triggering actions in a timely manner (automatic redirect, vacation messages…), and running a machine 24/7 with a mail client polling new mails every 30s is not a solution

We provide a much better way to do this using the SIEVE filters. Shortly, SIEVE is a language dedicated to expressing mail filters (also called rules). Our server is able to process your mails according to these filters as soon as they arrive. You then don't have to care about them anymore, and may use light mail clients or webmails when you're not on your machine with your favourite software.

Rules Configuration

You can express sort/reject/vacation/… filters using these rules, as the capabilities are very rich. Several softwares support managing SIEVE rules:

  • Icedove/Thunderbird:
    using the SIEVE extension (in xul-ext-sieve Debian package), it provides a rules editor (for power-users)
  • Roundcube:
    coupled with the sieverules extension, provides an easy to use web interface
    this webmail has been made available here
  • sieve-connect:
    provides a CLI to upload/download/activate your rules files
    this tool is available on shell hosts

(tell us if you know more software supporting this feature)

To push your filters on the server, a dedicated protocol exists: MANAGESIEVE (TCP 4190) on sieve.duckcorp.org. Our webmails are already configured to use it, but it you use sieve-connect from our hosts or your own mail software, you'll need these parameters.

You can read more info about SIEVE here:

Global Rules

Global rules are provided to ease configuration on specific filters. They can easily be included in your own configuration.

Available rules:

  • incoming_spam:
    SPAMs will automatically be delivered in the Junk folder and marked as important

If you write your own custom rules, here is an example on how to use one of them:

require ["include"];

include :global "<rule-name>";

Default Rules

The default settings (since 2112-02-11) are to use the global incoming_spam rule (see below), in order to provide a simple default configuration for most users. All other mails will end-up in your Inbox folder and you may then sort them by yourself.

If you create your own rules, the default rules won't apply anymore, so power-users can replace the default behavior completely to achieve what they really need. If you want to use the default behavior you can use:

require ["include"];

include :global "incoming_spam";

You can also take advantage of the antispam spaminess score to use a different threshold like this:

require ["fileinto", "spamtestplus", "relational", "comparator-i;ascii-numeric"];

# if SPAM score is >37% then move into the junk box
if spamtest :percent :value "gt" :comparator "i;ascii-numeric" "37" {
  fileinto "Junk";
  stop;
}

Fetching eMails from an External Mailbox

Something to do... TODO:

Shared/Public Folders

You may need to share mails or messages with friends or people you do stuff with (in a project or association). Depending on your needs, two solutions are possible:

  • share some of your own private folders, thus called shared folders
  • manage a special folder hierarchy, called public folders (even if they may not be accessible to everyone)
    if you need one, ask an administrator; the namespace name can be freely chosen but must be unambiguous and is subject to approval

Namespaces

Through IMAP, or our webmails internally using IMAP, it is possible to partition the folder hierarchy into namespaces. Traditionally you are using the root namespace for your private folders. Additional namespaces can be created and will appear among your own folders or separate, depending on your mail client's choice of representation. To avoid name clashes, we decided to prefix all additional namespace names with a #.

Since 2011-05-14, the following extra namespaces are created and reserved:

  • #Shared, containing all folders other users decided to share with you
  • #MilkyPond, containing public MP/DC informational mailboxes you may subscribe at will

Folders Permissions

Using IMAP, it is possible to setup rights (read only, write allowed…) to your own folders in order to share them with other users, or group of users.

Public folders are owned by no-one, and must be created by the administrators. Once your request is accepted we will delegate its administration to your care. It can then can be managed like shared folders.

Software Support

Client mail softwares support:

  • RoundCube:
    support namespaces, shared and public folders configurable via Settings->Folders, selecting a folder then using the Sharing tab
  • Icedove/Thunderbird:
    support namespaces, shared and public folders configurable via folder selection and Tools->Imap-ACL menu action

Most other softwares have namespace support only, so you should be able to use shared/public folders you have rights on but not configure them yourself (tell us if you know more software supporting this feature).

Limitations

Maximum Mail Size

Mail you send or receive are limited to 20MB. If you need to transmit much bigger data, then a mail transport is not appropriate, you'd better use a file sharing method instead.

Quotas

Even if it would be nice to live without it, we had to establish quotas to force people sort their mails out once in a while and delete useless things instead of leaving an ever-growing mess behind.

The default quota is 512MB which is not that big but should match needs of most users. This said, you may ask us for more and there's no reason we would refuse a reasonable demand.

Technical Details

This service is made using: