Tuesday, August 10, 2010

MailNews Sync 0.3.5 - Sync feed and newsgroup headers in Seamonkey

It's pencils down week in GSoC 2010 and MailNews Sync for Seamonkey has reached version 0.3.5 on AMO.

MailNews Sync 0.3.5

-Setting up MailNews Sync

You can install MailNews Sync 0.3.5 from here. It requires Firefox Sync addon which you can get from here (or you can get the development version I used for testing from here).
MailNews Sync engine has been set to true by default.

-Initial Sync

MailNews sync engine will sync all feed headers on first sync. However for syncing newsgroup headers there is a UI option 'Sync Message Headers' which can be selected from the right click menu of any newsgroup (please see image below). I have included this UI option because newsgroups may contains tens of thousands of messages and having all their headers synced up on initial sync will take a very long time. When sync option is selected for a newsgroup a preference 'mailnews.sync.accounts.fullupload' will be set with the newsgroup's name so that its headers will be synced up at next sync. At the same time another Boolean preference 'mailnews.sync.account.nntp.newsgroup_name' will be set to indicate that sync should be done for that newsgroup.

If sync doesn't run immediately as expected (the sync scheduler runs periodically depending on the amount of items that are in the queue waiting to be synced to the server) you may force sync to run immediately by clicking on the sync icon in the statusbar or by selecting the 'Sync Now' option in 'Tools' ->'Sync' menu.

If stop-script warning is encountered during sync please accept continue for now, until this is fixed in a future version.

-Up sync

Up sync happens in batches of 100 records at a time and can take sometime depending on the number of header records you are going to sync. In sync activity log (about:sync-log) a changed header item is indicated as follows,
Tracker.MailNews     DEBUG Changed item,
ID: 4AF3E769.60505@hfigge.myfqdn.de ,
FLAGGED: true , READ: false , TAG: $label2
This will be upsynced to server depending on whether or not duplicate record is present.

-Down Sync (includes caching)

Down will be indicated by a log record as follows,
Store.MailNews       DEBUG (10468)Down sync of mailnews header,
ID: 9pqdnYvr3Nr2GYLWnZ2dnUVZ_q1i4p2d@mozilla.org ,
URL: mozilla.support.seamonkey , FLAGGED: true , READ: true ,
TAG: [{"tagId":"$label2","tagName":"Work","tagColor":"#FF9900"}]

Store.MailNews DEBUG (10469)Down sync of mailnews header,
ID: 3IadnWFGE7zk_GrXnZ2dnUVZ_vSdnZ2d@mozilla.org ,
URL: mozilla.support.seamonkey , FLAGGED: true , READ: true ,
TAG: [{"tagId":"$label2","tagName":"Work","tagColor":"#FF9900"}]

If the feed of newsgroup of that header record is not available on the other pc, the header record will be cached in a file /weave/mailnews/mailnewsReSync.json. On every SM start-up it the MailNews Sync engine will search for those header feed/newsgroup subscriptions and apply them if available. Caching can be turned off using preference 'mailnews.sync.caching.enabled'

-Syncing Flags, Tags and Read status

MailNews Sync also supports syncing of custom tags and sync multiple tags applied for a message.

I also started work on feed and newsgroup subscription sync but since this was not origianlly planned for the project I will work on it later. There is also the possibility of extending MailNews Sync for syncing email headers. More details about the project can be found here.

Working with Seamonkey community has been really great so far. It's just a great developer community and #seamonkey is definitely one of the most funny and interesting IRC channels on moznet!:D Seamonkey developers work really hard but they really enjoy the work they do and they are very supportive of newcomers. I am especially thankful to KaiRo (Robert Kaiser) and Iav (Igor Velkov) for mentoring this project and also to everyone in the SM developer community who have promptly responded to my queries and helped me correct my coding errors etc. Hope to write another post about my full GSoC experience later on! There is still 5 days left for firm pencils down date and I hope to get some more testing and bug fixing done before that, then i'll be taking a 3 week break for my end semester exams. Afterward I will get back to SM to work on integrating MailNews Sync into comm-central along with the rest of sync code and to try out some C++ work for a change:)


  1. Hi!

    I haven't had a chance to really test this yet but it looks very promising.

    Just a quick note: Your changes seem to have broken the folder pane context menu which now shows all entries, whether they are enabled or not (e.g. Empty Trash should only be available for trash folders). Probably because you are overriding the default onpopupshowing event handler of the folderPaneContext in your account-sync-settings.xul (BTW: popup has been replaced by menupopup on trunk!). You can either set onpopupshowing in JS and prepend your JS call there (and not set it in XUL, the cleaner solution), or append the old value from mailWindowOverlay.xul, "return FillFolderPaneContextMenu();", to the value of your XUL attribute.
    If you have further questions, don't hesitate to ask (me, KaiRo, Mnyromyr, Neil, whoever you can catch on IRC). :-)

  2. Hi Jens,

    I made a mistake there. Thank you for pointing out the error:) I will check the solutions you have outlined and fix it.

    Also thanks for all the code reviews you have done for me so far:) and hope you will find some time to test MailNews Sync and give some input on possible improvements too..

  3. Went with option2 for now but need to switch to option 1 for a better fix later.

    Updated https://addons.mozilla.org/en-US/seamonkey/addon/207601/ and screenshots above.

  4. Seems to kind of work now, yes, but of course it's suboptimal since you're not taking the result of your setSyncMenuItem function into account for the whole onpopupshowing result (return value).

    Also is it correct that I can select Sync Message Headers on a news account itself (instead of only the newsgroups below it)? Do you support syncing the whole account in the back-end?

    I did some preliminary testing, basically just checking about:sync-log. I saw some real FLAGGED/READ/TAG updates for feeds, but for newsgroups I only saw "Sync for: (a NG) is true" even though I changed read states and tags. Is that correct? Or is that not implemented yet? What should appear for NGs, and when?

    We can move this to mail or IRC if you like. :-)

  5. Tnx Jens for all the feedback given so far!:)

    I'm noting down the following as needing further improvements..

    - Use xul command elements to add 'Sync Message Headers' option to message folder pane menu.

    - Check that the 'Sync Message Headers' option is always displayed only for the correct context (need to remove it from the newsgroup server menu).

    - Force newsgroup initial sync to happen immediately after the 'Sync Message Headers' option is selected for a newsgroup (right now it requires selecting sync option from sync menu or waiting for next scheduled sync to run).