C+Dav connector Beta

Status of the Software

Two-way sync of contacts and calendar events is implemented. Multiple calendars and addressbooks are supported. The calendars will also show up in webOS, so you can define different colors for them.

A complex mechanism for auto discovery is implemented. Servers are proped for a lot of different options. That “should” make configuration a bit easier. Additionally there are now multiple account templates for popular services like iCloud, Google or Yahoo where you do not have to worry about finding the right URL.

There is a C+Dav app now that has some functionality, you can trigger a “slow” sync, a discovery and also manually trigger the synchronization. This is only for convenience.

I will add the versions for upsync here, also, because I did not run into a lot of trouble with them during my tests. But: BE WARNED! I do not take ANY responsibility for lost data! Please be sure to have some kind of backup of your data (anything that your server or another device can import), before you use the upsync version!
Also: If you want to switch from downsync only to downsync and upsync version you should delete the account, switch the version and re-create the account. Otherwise webOS will do strange stuff. Really. Trust me here, please.

Required Patch

webOS has some issues with vCard processing, I patched these for all currently used 2.x webOS versions. Please select the right patch from the dropbox folder and install it. If you don’t do that you’ll get strange characters in contact data fields.

The patch will conflict with improved vCard export patch from patch feed. You can safely uninstall it, it is included in the patch files in the Dropbox folder.

Current Patch Version: 2.
To update the patch, you have to remove the old version. Luna restarts are not necessary.

General How-Too

Install the software fitting your device either using Preware, Quick Install or palm-install. I strongly advise first to test the downsync only version with new servers. Personally I only test on ownCloud, currently. All others are more or less untested.

After installation go to the Accounts app on your device and create a new account. You’ll see a bunch of new account templates starting with “C+DAV”. Select one of those matching your service or the general one. If there is a special account for your server, please use it. It  will make your life much easier.

In the next scene fill in all fields. For the generic one the URL is the most important one. If the server type is known and can be detected (for example the URL includes owncloud), then the software will take care of finding the right URL. If not, you need to supply an URL that “speaks” webdav (carddav and caldav rely on webdav), so we can use it to check the credentials and ask it for the users principal and home folders. For most servers it is not necessary to give a full calendar URL or something. Some examples:

  • owncloud: https://server/path_to_owncloud/remote.php/caldav
  • egroupware: https://server/path_to_egroupware/groupdav.php
  • Yahoo: https://carddav.address.yahoo.com
  • iCloud: https://p02-contacts.icloud.com:443

If credentials could be checked, you can create the account. webOS will now try to sync your data. Sometimes (quite often actually) this fails on the initial sync. You can manually (re-)trigger a sync using the calendar or contacts app or the C+Dav app (currently only on webOS phones). If that brings up all your data, you are fine and can test more functions.

A word on the account templates

The account templates exist for two reasons. On the one hand they make configuration easier on the other hand webOS does not allow multiple accounts of the same type with the same username, which can get a bit nasty if you use your e-mail address on multiple services. There are currently a number of account templates:

  • Google: This uses OAuth2.0 authorization (including support for two factor authorization), which is required for calendars now. Calendar & Contacts should work.
  • iCloud: Testing is hard, because it seems I can’t create a free iCloud account. No known issues, though.
  • Yahoo: The servers are strange. Calendar & Contacts should be working.

Trouble shooting and logs

If anything goes wrong, please contact me and maybe already attach a log. Logging goes to the system log ( in /var/log/messages ). You can create a log using Lumberjack. Please use the custom filter for “cdav”. The nicest way is to select “Follow log” and then trigger a sync or discovery where I can see the errors (probably by creating events / contacts, if you have issues with upsync).
Another possibility is to copy the  file from the device and cut the relevant parts from it or use the command line like this:
cat /var/log/messages | grep cdav > /media/internal/cdav.log
This will create a cdav.log file in your media partition which you can e-mail me.

If you want to clean your device, please follow these steps in the right order:

  1. Delete all c+dav accounts in the device manager
  2. Reboot the device
  3. Uninstall the (old) version (for example via Software manager)
  4. Please use Impostah to clean the on device database, to do this, start Impostah, select databases, then on the upper field select “Persistent”.
  5. From the second list first select org.webosports.cdav.account.config and then select “Delete all” from the app menu.
  6. Do the same for org.webosports.cdav.account.calendar and org.webosports.cdav.account.contacts.
  7. After that, do the same for the other org.webosports.cdav.* you find in the list
  8. Go to Impostah => Acitivities, select “Service persistent” in the first field
  9. Look for acitivites with “org.webosports.cdav” in their name, select them and remember their activityIds.
  10. For each of those ids run this command from the command line:
    luna-send -n 1 -a org.webosports.cdav.service palm://com.palm.activitymanager/cancel '{"activityId": X}'

    Where X should be replaced with the activityId.

Sometimes these steps are necessary to make sure that you have a clean start. Otherwise you might have phantom events/contacts remaining in the database. Usually all of this should happen automatically. But there might be bugs in those procedures.

Important notes

Only webOS 2.2.4 and webOS 3.0.5 (and open webOS by webOS ports) are supported. If you have a Veer with 2.1.X, you can try to patch your sync framework to the version of 2.2.4. It worked for me on a Veer.

Also, please tripple check that you do not have the patch “System: Sync fix for stable upload” installed. It WILL run the connector in an endless loop eating up all your battery and your data volume quite fast!

The first sync can last very long, especially if the account has many entries on the server and the connection is not very stable. Please leave it for some time, best would be overnight on a good WiFi.

Current version

0.3.18, get the files via Dropbox.

Only webOS 2.2.4 and webOS 3.0.5 (and open webOS by webOS ports) are supported. If you have a Veer with 2.1.X, you can try to patch your sync framework to the version of 2.2.4. It worked for me on a Veer.

Also, please tripple check that you do not have the patch “System: Sync fix for stable upload” installed. It WILL run the connector in an endless loop eating up all your battery and your data volume quite fast!

What file do you need?

  • TouchPad, only downsync: org.webosports.cdav_*_all_enyo_no_upsync.ipk
  • TouchPad, two way sync: org.webosports.cdav_*_all_enyo_upsync.ipk
  • Phones, only downsync: org.webosports.cdav_*_all_no_upsync.ipk
  • Phones, two way sync: org.webosports.cdav_*_all_upsync.ipk
  • Patch for webOS 2.1.*: MojoTransportAndSyncFramework_21*to244.patch

 

Changelog

0.3.18:
o fixed issues with db query limit of 500 objects
o fixed issue with URI encoding that affected Yahoo.
o fixed crash on empty ics retrival
o changed some stuff to allow automated testing (still in early stage)
o fixed google resolving (non critical issue)
o now additionally writing log to /media/internal/.org.webosports.cdav.service.log in order to find crashes
o Retry etag retrival without filtering, if filtering is not supported.
o If upload fails with error code that hints for impossible upload, do not retry upload.
o Fixed minor issues with proxy usage
2014-10-15: Achim Königs <garfonso@mobo.info>

0.3.17:
o fixed check credentials issue for yahoo accounts.
2014-09-11: Achim Königs <garfonso@mobo.info>

0.3.16:
o first working mdigest support implemented
o fixed change credentials on webOS 3.x and LuneOS
o categories are now stored for contacts, too and send back to server, so they do not get deleted
o imporved network code for webOS 3.x and LuneOS
o implemented retry mechanism if upsync fails. Prevents deletion of failed upsyncs.
o Allows to ignore certificate errors on webOS 3.x and LuneOS
2014-09-10: Achim Königs <garfonso@mobo.info>

0.3.15:
o fixed issue with contact export in some cases
o reworked authorization a lot to make md5 digest work (untested)
2014-08-02: Achim Königs <garfonso@mobo.info>

0.3.14:
o fixed issues with contact import (please install newest contacts patch also, you’ll need to uninstall old one!)
o reworked authorization a bit, maybe md5 digest works now.
o minor fixed & imporvements (mostly LuneOS)
2014-07-12: Achim Königs <garfonso@mobo.info>

0.3.13:
o fixed deleting accounts
o added uri component decoding where necessary, fixes some issues with virtual google-events (like weather or holydays)
o do slow check next time, if download of single items failed.
o go on if single addressbook / calendar fails (relevant for current bug in owncloud 6.0.3. with contact birthdays)
2014-07-04: Achim Königs <garfonso@mobo.info>

0.3.12:
o changed getting username from google, now retrieves e-mail address and works with google plus deactivated, too.
2014-06-30: Achim Königs <garfonso@mobo.info>

0.3.11:
o removed wrong unescaping in vCard processing.
2014-06-27: Achim Königs <garfonso@mobo.info>

0.3.10:
o fixed google login issue on TouchPad
o renamed google & yahoo accounts
2014-06-27: Achim Königs <garfonso@mobo.info>

0.3.9:
o made newer http.request method work on TouchPad, too (previously was tailered only to OWO)
o fixed an issue where the service sometimes deleted just upsynced objects from device
2014-06-11: Achim Königs <garfonso@mobo.info>

0.3.8:
o reworked retry mechanism a bit (still not sure why that is needed at all)
o fixed some more issues with recurring events
o fxied issues with no events at all
o added filter to get events only one year back (do I need to make that configurable?), future should still be included completely
o fixed another possible sync-endless loop
o fixed issue with ical parsing if no event was received at all
2014-06-10: Achim Königs <garfonso@mobo.info>

0.3.7:
o fixed issue with upload
o sync on edit now only syncs the edited capability
o implemented proper exception handling for recurring events
o fixed ical parsing error
o reworked timezone handling in ical parsing, should prevent NaN-errors in calendar
o accept delete as succesful if server can’t find object
o prevent possible rev-errors in put db, might have caused issues with synconedit syncs
o some code refactoring in syncassistant.
2014-06-09: Achim Königs <garfonso@mobo.info>

0.3.6:
o fixed possible sync-endless loop because of wrong sync on edit activity creation
2014-06-03: Achim Königs <garfonso@mobo.info>

0.3.5:
o fixed possible dns issue that could hang the service for ever.
o optimizations in loading routine.
2014-06-03: Achim Königs <garfonso@mobo.info>

0.3.4:
o fixed possible connection issue that could hang the service for ever.
2014-05-30: Achim Königs <garfonso@mobo.info>

0.3.3:
o fixed possible crash in service with brittle network connections
o fixed contact sync not working after reboot
2014-05-29: Achim Königs <garfonso@mobo.info>

0.3.2:
o fixed error in checkCredentials that prevented return on network / authentication errors.
o more fixes to timing out messages, reworked message timeout & retry mechanism
o fixed issues with initial sync and account deletion (please clean up database and activities manually!)
o fixed issue with periodic sync when only one of multiple capabilities is disabled
o fixed issue with periodic sync only syncing contacts (you need to disable ony capability and re-enable it to solve this issue)
o now correctly prevents sync if account is being marked for deletion.
o fixed issue with account creation if contacts were not enabled
2014-05-29: Achim Königs <garfonso@mobo.info>

0.3.1:
o fixed: auto detection could screw up urls on some servers.
o some fixes to timing out messages, reworked message timeout & retry mechanism
o updated icon
o fixed a bug in ical conversion that could prevent calendarevents from showing up
o added simple proxy support, use http_proxy environment variable to use. (No https connect support, i.e. proxy will be able to read everything and has to handle the https connection)
o removed text-box in google-cred-ui and ask google for display name.
2014-05-28: Achim Königs <garfonso@mobo.info>

0.3.0:
o added enyo app and moved check-credentials-UI into sub folder.
o created enyo UI for google OAuth2.0 access
o created mojo UI for google OAuth2.0 access
o made service ready to work with OAuth2.0 tokens.
o added onenabled assistant to delete sync on edit activities on disabling.
o fixed issue with account creation.
o removed google.contacts account (you can still use it with general account and url www.google.com)
2014-05-11: Achim Königs <garfonso@mobo.info>

0.2.9:
o fixed issue carddav/caldav namespaces which lead to issues during discovery and sync for some servers
o fixed issue with empty / VTODO calendar entries which could make the service stuck in sync mode
o fixed content type for vcard/vcalendar downloads
o enabled URL scheme for yahoo
o fixed issue that sometimes error state was forgotten on a failed sync
o changed contet type of xml requests to text/xml.
o added yahoo account
o fixed possible hang in checkCredentials.
2014-05-08: Achim Königs <garfonso@mobo.info>

0.2.8:
o fixed issue with iCloud folder parsing.
o fixed possible endless loop with sync-on-edit.
2014-05-04: Achim Königs <garfonso@mobo.info>

0.2.7:
o fixed error display in mojo validator app.
o added url scheme for SOGo server
o reworked url resolving to reduce case manipulation as much as possible
2014-04-11: Achim Königs <garfonso@mobo.info>

0.2.6:
o fixed possible issue with deleting multiple calendars/addressbooks
o fixed issue with folder uris on some servers.
o added workaround for wrong URIs returned by SOGo server
2014-04-11: Achim Königs <garfonso@mobo.info>

0.2.5:
o fixed issues with case in URLs.
o changed url resolving logic a bit. Method now returns false on failure, not original URL
2014-04-11: Achim Königs <garfonso@mobo.info>

0.2.4:
o fixes for iCloud compability
o made egroupware calenderevents no duplicates hack optional and activate it when egroupware is detected (TODO: implement something to add this manually)
o prevented possible endless loops if server returns corrupted ctags or etags.
o made stuff more robust.
o added google account template to prevent username issue.
o added icloud account template to prevent username issue.
o made sync more memory efficient on big datasets.

2014-04-10: Achim Königs <garfonso@mobo.info>

0.2.3:
o fixed issues with some calendar events
o urlschemes improved to allow multiple keys, will allow support for hosted owncloud systems
o changed upsync logic to be more robust.
o a lot of fixes in vCard processing which should allow contact upsync in future.

2014-03-02: Achim Königs <garfonso@mobo.info>

0.2.2:
o Added messages to enyo-app, that it is not to be run manually, at this point in time
o Added another stage to mojo-app to allow some recovering stuff like trigger slow sync.
o Cleaned up auto discovery and added possible support for iCloud.
o Changed host/port/protocol stuff in Caldav.js, now should work fine with multiple servers
o Caldav.js now also reacts to connection issue with server other than send timeout.
o Added urlscheme.js as place to store known server configs in. These do not require a discovery anymore, fallback is implemented, though.
o Prevent local duplicates if upload (or download of etag) did not work.
o Prevent endless loop during redirections to always the same URL.
o Download 10 items at one time, which keeps memory footprint low even for massive downsyncs.
o Changed back sync activity to “sync” which will sync contacts and calendar on periodic sync
o Overwrite “complete” in syncAssistant and allow installation of multiple “SyncOnEdit” activities.

2014-01-04: Achim Königs <garfonso@mobo.info>

0.2.1:
o Initial test versions send to some testers
o Fixed issues with periodic sync
o service assistants renamed, now single assistants for contacts and cards
o Fixed parsing issues with some caldav server

2013-12-21: Achim Königs <garfonso@mobo.info>

0.2.0:
o Addition of CardDAV as well as CalDAV support
o Use of mojo sync framework for regular syncs
o Code for two way sync but only downsync enabled by default
o Courtesy to Garfonso for this major push forward

2013-XX-XX: Stefan Schmidt <stefan@datenfreihafen.org>

0.1.0:
o Initial version (Not functional yet).
o Synergy account connector based on Plaxo example from HP documentation.
o Custom validator UI scene to allow an URL input field and maybe more options later.

2013-08-10: Stefan Schmidt <stefan@datenfreihafen.org>

One thought on “C+Dav connector Beta

  1. He did a fantastic job, it works perfect with owncloud on my touchpad, on pre3 I will test next weekend, and then google is gone for ever.

Leave a Reply

Your email address will not be published. Required fields are marked *