Re: [coldsync-hackers] Anybody up for a perl sync conduit?

[Christophe Beauregard <christophe dot beauregard at sympatico.ca>]

On Saturday 20 December 2003 21:04, IwantToKeepAnon wrote:

> As it stands, a db can only be syncd once.  After
> that, all the flag information is lost and a second
> sync would do nothing.  So your first case scenario
> doesn't happen today (for sync, not dump or fetch).
> Am I correct?

The flags are only cleared when they're explicitly cleared via 
dlp_CleanupDatabase. The generic sync does call this as part of its 
cleanup. A few other conduits which think they're "in charge" of a 
particular database might also do this (The Zire71 photo sync?)

What this means is that sync conduits generally need to run _before_ the 
generic sync if they need to know about dirty records. If the conduit 
expects the generic sync to do its usual magic (which it probably should), 
the conduit just doesn't do a dlp_CleanupDatabase.

I'm not sure this is really heavily explained anywhere. Dump and fetch 
conduits are fairly well documented, but sync conduits still seem to be 
very misunderstood.

> There should be a disclaimer/warning to not create a
> conduit that performs complex and long running
> operations.  E.g. don't create a MailDB conduit that
> sends mail via SMTP or the Web.  If these extra
> resources are unavailable, then the PDA could be kept
> online for too long or until a timeout occurs causing
> a sync failure.

There's already pretty good warnings about this sort of thing. That's the 
whole rational for the dump conduit.

I've been abusing this a lot lately with some of my conduits, though. 
Unfortunately, a lot of the really interesting stuff can only be done with 
the Palm online... Being able to run things like fetch conduits in batch 
(cron, etc) might help, here, but there's no provisions for database 
locking in coldsync, so that's incomplete functionality.

> Let me know your thoughts ...

Personally, I like the idea. There's a whole pile of record-oriented conduit 
actions while would be handled nicely through something like this. Sending 
mail, archiving, etc. Certainly the generic sync could be written in terms 
of this framework, although I wouldn't go so far as to actually remove the 
existing generic sync. My approach would probably be to embed your 
framework into it via perl XSUB calls. There's a strong argument to be made 
that the underlying PDA-to-local-database sync logic _shouldn't_ be too 
easily messed with by perl hackers. XSUB introduces too much dependency on 
perl itself though, so it's not really a good idea at this point.

Also, your framework won't solve all the problems... sync conduits, in my 
relatively limited experience, can be broken into two categories:

1. record oriented syncs. Mostly these run as sync conduits because they 
need to get status information back to the PDA ASAP. i.e. sending mail 
doesn't _need_ to be done with the Palm online, except that the user likes 
to know after the sync if the mail actually got sent. Your framework will 
handle this class of problem nicely and, I think, better than having 
multiple conduits running sequentially before the generic sync.

2. database oriented syncs. I've been working on a few of these, and the 
short summary is that some syncs need to perform complicated operations 
across the entire database (or, in the case of Izzie's Titrax conduit, 
multiple databases). My tv-sync, for example, expires records, calculates 
the latest TV record, and if necessary pushes new listings back to the PDA. 
I don't think your record oriented framework will handle this nicely, not 
based on the information you've published so far.

c.
This message was sent through the coldsync-hackers mailing list.  To remove
yourself from this mailing list, send a message to majordomo@thedotin.net
with the words "unsubscribe coldsync-hackers" in the message body.  For more
information on Coldsync, send mail to coldsync-hackers-owner@thedotin.net.