PRC-Tools, Multiple Segments and Destructors

This post will only be of interest to hardcore PalmOS programmers.

I’ve been fighting mysterious errors in a project that uses PRC-Tools’ gcc to compile C++ into multiple segments. Sometimes (but not always), trying to delete objects that had destructors defined in segments other than the default would cause code to jump into a (deterministic but) effectively random function. Pulling the dtors out into the main segment caused them to execute correctly, but moving all destructors into the main segment wasn’t really an option b/c I am already near the size limit in my main segment. With a little help from Google, I reasoned that using -O2 might be the problem, and that appears to have been the case.

So, it looks like PRC-Tools 2.3 has a bug when emitting code to call destructors in other segments when optimizing. Be warned. To be safe, I’ve moved to using -O0 (no optimization) for almost all of the files in my project, except for a critical few that do things like database sorting, where I use -O3 and am careful not to use much more than the C subset of C++.

Palm Resource Workaround

Newer versions of Palm’s SDK recommend an XML-based resource description format (XRD). This is good news, because it makes it easy to write tools to help work around some of the problems associated with PalmOS resources.

A problem I recently faced was creating multiple instances of a form based on a single resource. Each instance had a different C++ class in which its code was contained, but the control layout was identical and should stay so through maintenance changes, so I didn’t want to make three different copies of the resource and have to maintain all three. However, the Palm Forms API is designed to only load one instance of a form with a given ID at a time, and each resource has exactly one ID.

The solution I hit upon was to create an XSLT stylesheet that knows how to pass most elements through, duplicating a few of them. A separate XML control file specifies which elements to duplicate and what their IDs should be. The output is written to a temporary file, and that temporary file is fed to palmrc instead of the original. Now I can pick a few forms that I need to load multiple instances of and duplicate them as many times as necessary.

Here’s what the control file looks like.

<?xml version="1.0" encoding="utf-8"?>
A control file for use with ResourceDup.xslt. Specifies and controls
compile-time resource duplications, allowing a single resource maintained
in an XRD file to become multiple resources in a compiled PRC.
<res -dup-control>
<!-- Main List Forms -->
<!-- List1 = 1600 -->
<dup id="1600" new-id="1601" /> <!-- List2 -->
<dup id="1600" new-id="1602" /> <!-- List3 -->
<!-- Main Edit Forms -->
<!-- List1 = 1700 -->
<dup id="1700" new-id="1701" /> <!-- List2 -->
<dup id="1700" new-id="1702" /> <!-- List3 -->

I also just ran across this: I haven’t used it yet, but I am likely to. It allows you to generate a header file containing symbolic names for your resources, rather than referring to them by number in code or maintaining your own mapping in a header file. That tool is probably a temporary solution, though: tools existed for the older resource formats that could generate header files, and I think it’s likely that the lack of any such tool for XRD files in the SDK is just an oversight that Palm will likely fix sometime.

Finally, I can verify that it is possible to use the new Windows-only Palm development tools on Linux, to some extent, using
wine. Palmrc works perfectly, but I haven’t been able to get pacc to work; it shows its logo banner, but then silently exits with no error messages. The utility of pacc on Linux is of dubious value right now anyway, since so much of the rest of the development suite is Windows-only (Palm! We want our Linux tools!) But getting palmrc and prcmerge to work means that I can use the newer XRD resource files without being stuck on Cygwin. I put the following in my current project’s Makefile to make this work.

UNAME :=$(shell uname)

ifeq ($(UNAME),Linux)
# Linux Settings
PRCMERGE :=wine -- ~/prc-bin/PalmOSTools/PRCMerge.exe
PALMRC :=wine -- ~/prc-bin/PalmOSTools/PalmRc.exe
# Cygwin Settings
PRCMERGE :=prcmerge
PALMRC :=palmrc

HotSync Woes

I’ve been doing a lot of research lately regarding PalmOS HotSync. My findings are rather frustrating. I’ve been trying to implement synchronization between a Palm with a built-in wireless network connection and a remote server. An appropriate conduit (= code implementing synchronization logic) has existed for the application for quite a while now for “standard” HotSync (= using a serial or USB cable to sync a Palm with a Desktop). That conduit has been well-tested and in use long enough to be trusted. The best of all worlds would be to put that conduit on the server and use Palm’s existing network HotSync capability to talk to it.

The Palm HotSync infrastructure assumes that a Palm belongs to a single user, and that the single user HotSyncs (synchronizes data) with some number of desktops. Of those desktops, exactly one of them can be designated the primary, which means that when a Palm tries to HotSync across a network, it will contact the primary desktop and sync with it over the network instead of a cradle. Setting this up is an all-or-nothing thing, though. You can’t have different primary desktops for each application. That means that a user can’t set up a network HotSync with a specific remote server without changing the same HotSync preferences used for all other applications on the Palm as well.

Now, normally this is what you want. In fact, it’s what you want so often that people who are knowledgable about Palms and HotSyncing may be wondering whether there’s something I just don’t get since I’m even talking about it. Really, though, what I need for this application is to have a HotSync profile that could be invoked from within my program on the Palm that caused a HotSync to occur using whatever the current network settings are, regardless of how the user’s “normal” HotSync is set up. I.e., if they hit a “sync” button within this application, I want to synchronize only that application, and I want to do it with a remote server. I could implement a custom TCP protocol (and in fact suspect that’s what I’ll have to do), but ideally I could just set up a HotSync profile programatically and tell it to run. At the very least, I should be able to save the user’s current “normal” HotSync settings, change them to be network HotSync to a given server, invoke a HotSync (that, ideally, gives status updates back to my application, but could show a HotSync dialog instead), then changes the user’s normal settings back after it runs. It would be a little dangerous (what if the device crashes before the user’s normal settings are restored?) but it would be functional.

All synchronization logic happens on the PC side of a Palm->PC connection, no matter what the physical link (USB, network, whatever). The remote server would, presumably, only contain a conduit for the one application in which it’s interested, so any other data on the user’s handheld would not be affected. There could be many different HotSync profiles on the user’s device: the one that they use for most syncing, plus one for each application that wants to synchronize with a specific remote server, if only the API were there…

Unfortunately, Palm doesn’t document an API or a database format for working with either network settings or HotSync settings. I’ve done a little bit of reverse engineering on it, but I’m afraid my kung fu may not be strong enough to figure out everything I need to know in a reasonable amount of time. One guy seems to have already done all or most of the work, but he charges US $500 for the source code, which is just a little extreme, at least for a small shop, to get access to information that Palm really should be providing as part of their API.

Really, what is Palm afraid of? If they would just document the damn functions, a whole new class of HotSync-using applications would become possible. They released an unsupported library a few years back that gave programmatic access to network settings, but not to HotSync settings, but they let it fall out of sync until, by OS 4.1, it no longer functioned. Currently, it looks like the only way to write a perfectly legitimate app like this is to place a significant configuration burden on your users, roll your own custom solution, or shell out the $500 to get a reverse engineered support library that is not guaranteed at all to work past OS 5.4 (OS 6 simulators are out, but the web page doesn’t mention whether this library works with m68k devices when run in OS 6’s 68k emulation environment).

I can somewhat answer my own question regarding what Palm is afraid of. If applications can start running their own HotSyncs programmatically to whatever servers they please, sychronization points could start to proliferate. You might have to start doing separate synchronization steps for each of your applications, instead of a single step for one. This is a major usability issue, and I totally agree that it would be a bad situation. On the other hand, you have cases like the app I’m working on. We already have a conduit that will always run when a normal HotSync occurs. However, it’s important that users be able to get updated data while they’re on the road, away from their desktops, when they may or may not have a laptop handy. We just want to run the same conduit we would during a normal HotSync, but we don’t want to force our clients to make their personal desktops available through whatever firewalls they may have in order to enable network HotSync, and we don’t want to force them to purchase a set of licenses for an expensive commercial network HotSync solution, nor do we want to purchase one ourselves if we can avoid it, for various reasons.

One possibility I’ve looked into is trying to get a small DLP (= the link protocol used by HotSync; actually it has a few companion protocols as well, but I’ll us DLP to refer to them all) server like coldsync to compile and run on the device. Coldsync is one of the many available alternatives to Palm’s HotSync software, intended mainly to run on Unix OSes, but compilable on Windows as well (using Cygwin). From an initial inspection, it looks like Coldsync is capable of running either end of a network HotSync (either client or server), so if I can hack it to run on a Palm and talk to actual Palm databases, it would be like having a custom network HotSync emulator built into the app. Using the system’s would still be preferable, though. Another option would be to forget about DLP entirely and do a custom protocol over TCP connection, implementing just the bare minimum of DLP-like messages necessary to get this app to work. That’s a lot of work, too, for something the system could be doing for me if only the damn APIs were documented.

On the server side, things are just slightly better. Palm’s HotSync software is meant to sit in a logged-in user’s system tray. It creates dialogs and waits for user input, and it is not meant to be used with an arbitrarily large number of remote network users, so it’s not suitable for use as a system service (= daemon, = app that sits around all day and accepts HotSync requests from arbitrary users over the network and never tries to talk to a local user b/c there probably isn’t one). There are some commercial alternatives, and there are some open source alternatives. The open source alternatives are very attractive (especially coldsync, which seems especially elegant to me from what I’ve seen so far), but their downside is that they do not work with applications written to the Palm HotSync API. They implement their own APIs for implementing conduits. So, in order to get an existing API to work with one of the open source servers, I’d have to write a bridge. That’s a task I can handle (and which might be useful to other people, once done), but it’s still frustrating. Palm’s been really (really, really) slow dealing with the issue of Enterprise Synchronization (which is really what all this boils down to): the first Palm came out in about 94-96 time frame (not sure exactly), and by 2005 one would think that viable multi-user HotSync would be part of the PalmOS infrastructure, not a high-cost, high-frustration extra.