Mailsync

bench get-app https://github.com/EffizienzAgent/mailsync

[![Listed on Frappe Gems](https://frappegems.com/api/method/frappe_gems.seo.badge?app=EffizienzAgent%2Fmailsync)](https://frappegems.com/gems/apps/EffizienzAgent/mailsync)

IMAP mail sync for the Frappe Framework

Mailsync keeps Frappe's Communications in step with your IMAP mailbox:

• One login per account. Every two minutes one background job per Email Account logs in once and runs the inbox pull, the Sent folder import and the status sync over the same connection.
• Sent folder import. Mails you send from any client (Apple Mail, Thunderbird, your phone) show up as Sent Communications. Mails that Frappe sent itself or already knows are skipped.
• Status from the mailbox. A received mail becomes Replied when a reply sits in the Sent folder, and Closed once it is moved out of its folder (archived or deleted).

Mailsync only reads the mailbox. It never moves, flags or deletes mail and never sends email.

Installation

bench get-app https://github.com/EffizienzAgent/mailsync --branch version-16
bench --site  install-app mailsync

Compatibility: Frappe v16. The version-16 branch always points to the latest release; tagged releases and the changelog are on the releases page.

Set up an Email Account with IMAP and Enable Incoming as usual. Mailsync picks it up on the next run.

How it works

mailsync.mail_sync.sync_all runs every two minutes. For each incoming account it enqueues one job on the default queue with a 600 second timeout. The job name is Frappe's own pull_from_email_account|, so Mailsync and Frappe's stock pull never run at the same time for one account. IMAP accounts get mailsync.mail_sync.sync_account. POP3 and Frappe Mail accounts get Frappe's stock pull.

sync_account logs in once and hands the open connection to three steps. A failing step is logged and the other steps still run. If imap_timeout is not set in the site config, it is set to 60 seconds for the job.

1. Inbox pull. Frappe's own EmailAccount.receive, reusing the connection.
2. Sent folder. The folder is found by the \Sent flag, then by the account's Sent Folder Name, then by common names (Sent, INBOX.Sent, Sent Items, Gesendet, ...). On the first run Mailsync only remembers the current position and imports nothing older. After that it imports up to 50 new mails and up to 100 MB per run. Mails over 25 MB are skipped and logged. A mail is skipped when its Message-ID is already a Communication, is in the Email Queue or was generated by the site. If the folder's UIDVALIDITY changes, Mailsync starts again from the current position. The position is stored as a default named mailsync_sent_folder:.
3. Status. Open and Linked received mails from the last 14 days are checked. Mails whose Message-ID has characters outside printable ASCII, spaces, quotes, backslashes or angle brackets are left alone. If the server rejects a search, only the mails in that search keep their status. A reply in the Sent folder (by In-Reply-To or References) sets Replied. A mail that is no longer in its IMAP folder is set to Closed. Closed mails are never reopened.

TLS

Python's imaplib and smtplib do not check the server certificate unless they are given an SSL context, and Frappe passes none for IMAP STARTTLS, SMTP STARTTLS and SMTP over SSL. Mailsync fixes this in every request and background job, including Frappe's own inbox pull and the outgoing email queue. These connections now check the certificate and the host name against the system's trusted CAs. IMAP over SSL was already verified by Frappe.

For a test server with a self-signed certificate, you can turn the check off for one site:

bench --site test.site set-config mailsync_allow_self_signed_starttls 1  # incoming IMAP
bench --site test.site set-config mailsync_allow_self_signed_smtp 1      # outgoing SMTP

Do not set these in production.

Hooks

Other apps can extend Mailsync in their hooks.py.

Hook arguments come from mail headers and the IMAP server, so treat them as untrusted strings. Use them in database filters and escape them before rendering them as HTML. Never pass them to a shell, eval or raw SQL.

mailsync_recipient_links

Adds timeline links to imported Sent mails. Called once per recipient in To and Cc, except the account's own address and system users. Return a list of links.

mailsync_recipient_links = ["myapp.mail.recipient_links"]


def recipient_links(address: str) -> list[dict]:
    customer = frappe.db.get_value("Customer", {"email_id": address})
    return [{"link_doctype": "Customer", "link_name": customer}] if customer else []

Frappe still links known Contacts on its own.

mailsync_status_changed

Called after the status sync changed Communications, with a dict of Communication name to the new status ("Replied" or "Closed").

mailsync_status_changed = ["myapp.mail.status_changed"]


def status_changed(changes: dict[str, str]) -> None:
    frappe.publish_realtime("inbox_changed", after_commit=True)

Development

Tests run on a bench with a test site:

bench --site test.site set-config allow_tests true
bench --site test.site run-tests --app mailsync

The tests in test_imap_server.py talk to a real IMAP server. They run when MAILSYNC_TEST_IMAP_HOST (and optionally MAILSYNC_TEST_IMAP_PORT, default 3143) is set. CI uses GreenMail:

docker run -d -p 3143:3143 -e GREENMAIL_OPTS="-Dgreenmail.setup.test.all -Dgreenmail.hostname=0.0.0.0 -Dgreenmail.auth.disabled" greenmail/standalone:2.1.5
MAILSYNC_TEST_IMAP_HOST=127.0.0.1 bench --site test.site run-tests --app mailsync

The tests in test_tls.py and test_smtp_tls.py need IMAP and SMTP servers with a self-signed certificate. They run when MAILSYNC_TEST_STARTTLS_HOST (IMAP STARTTLS on port 31143) and MAILSYNC_TEST_SMTP_HOST (SMTP STARTTLS on 31587, SMTP over SSL on 31465) are set. CI uses Dovecot:

docker run -d -p 31143:31143 -p 31587:31587 -p 31465:31465 -e USER_PASSWORD=secret dovecot/dovecot:2.4.1
MAILSYNC_TEST_STARTTLS_HOST=127.0.0.1 MAILSYNC_TEST_SMTP_HOST=127.0.0.1 bench --site test.site run-tests --app mailsync

To check that a CA-signed server still connects, set MAILSYNC_TEST_CA_SMTP_HOST to a public SMTP host. This test needs network access and does not run in CI.

Professional setup & support

Need help setting this up or integrating it into your Frappe stack? Write to oss@effizienzagent.de (English or German). German-speaking SMEs: see mittelstandshub.de.

License

MIT, see license.txt.