Well this is going to be a bit of a ramble as opposed to an articulate article. It comes from my frustration with Android’s bugs and Google’s seemingly unwillingness to respond. If one takes a look at

http://code.google.com/p/android/issues/list

one can see a number of long lingering bugs which have not been addressed yet. Sure, loads of bugs have been fixed and Android continues to improve. I’ll come back to “improve” below. My gripe is with the bugs, which have been around for a while, that break functionality or standards.

For instance – proper handling of IMAP folders, IMAP drafts, IMAP IDLE support, Exchange sync if you delete e-mail messages while disconnected from your Exchange server, IPSEC L2TP PSK VPNs broken. These are bugs in features that are needed for mainstream business use in a smartphone. Sure not all consumers will run into all these bugs but with the recent #PRISM leaks and continuing concerns around privacy I’m sure more people will start using “their own” (for some definition of own as opposed to BIG providers such as Gmail/Hotmail/etc) IMAP servers and IPSec VPNs.

Business users will typically either use Exchange or IMAP (OK there are still some Lotus Notes users too – ever seen that mentioned on a stock Android device??) coupled with VPNs.  Both of these key features are crippled or lacking in one or more ways. Additionally, Android’s handling of recurring calendar appointments is poor to lacking resulting in potential frustration for business users. I guess haves “issues” in these area and offering a working Gmail app plays right into Google’s plan of owning everyone’s data and obviously mining said data for ad revenue!!!

Android is no longer a “new” operating system and has been through various updates along the way. Simple things like working, reliable e-mail should “just be there”. One should not need to install app (K9 mail for instance) after app (OpenVPN) to get working functionality that is supposedly built in to the OS. As Android ages it appear to be more of a way to make more money for Google than a concerted effort at a proper phone OS.

  • “improve” – yes new releases of Android come out every year or two with new wizzy-flashy graphics to dazzle the eyes. But come on – lets get the basic underlying functionality fixed before rolling out even more buggy code!

 

Well, I’m still vetting my HTC Sensation to see if it is a worthy replacement for my BlackBerry… Still not convinced as it happens. However, I have resolved one of my problems with it…

IMAP sent/trash/draft folders don’t work properly if you have a “typical” IMAP server with a folder prefix of “INBOX.”. The IMAP NAMESPACE RFC (RFC2342) defines how this works and how clients should handle it. As usual, Google developers figured they knew better than or just chose to ignore the RFC! As a result, the Andoird mail application doesn’t support folder prefixes, let alone automatically working correctly based on the namespace command.

If you are lucky and are using a IMAP server which does not have a personal namespace (Dovecot??) you might find the mail app works OK for you.

You can use the Android K9 mail client. I briefly tried it and it seemed to work. However, it doesn’t seem as “integrated” as the stock client so I gave up on it. It’s not a bad app – it just didn’t grab my attention. Luckily, there is a work around which is partially documented on the Android issues page at (thanks to those who posted there as it got me working on this workaround):

http://code.google.com/p/android/issues/detail?id=1811 in particular comment 83 and comment 84.

The work around is to reconfigure the mail application’s SQLite database to point to IMAP provided folders rather than the default folders which do not work. On the surface it is pretty easy if your phone is rooted – no idea how I’d do this if the phone was not rooted.

The method I used in the end, to get around the SQLite database files being in use, was to boot to clockworkmod-recovery, mount /data, adb the necessary files to the PC, edit the datafiles using SQLite and push them back using ADB.

The updating of the database is the trick. First, the ID of the account to be “fixed” needs to be determined. This account ID is then used to get a list of corresponding folders’ IDs from the mailboxs [sic]. These IDs and names are then used to update the accounts table which is the “magic bit”.

Here is the code snippet of the steps I followed. If you have a vague idea of technical stuff in this area, you should be able to follow the steps.

**boot into clockworkmod recovery
** mount /data on the phone
** connect USB cable to phone and PC and select HTC Sync mode
** check device can be seen by adb:

adb devices

** copy the data files from the phone to your PC (might not have the -shm and -wal files):

adb pull /data/data/com.htc.android.mail/databases/mail.db
adb pull /data/data/com.htc.android.mail/databases/mail.db-shm
adb pull /data/data/com.htc.android.mail/databases/mail.db-wal

** run sqlite on the data files

sqlite3.exe mail.db
sqlite>  pragma wal_checkpoint;            -- to clear the sqlite logfiles...
sqlite>  pragma wal_checkpoint(FULL);      -- to clear the sqlite logfiles...
sqlite>
sqlite> .headers on       -- to make it easier to see whats happening
sqlite> .mode column      -- or .mode list

sqlite> --view the current folders for all accounts
sqlite> select _id, _name, _sentfolder, _sentfoldertext, _sentfolderid from accounts;
sqlite> select _id, _name, _trashfolder, _trashfoldertext, _trashfolderid from accounts;
sqlite> select _id, _name, _draftfolder, _draftfoldertext, _draftfolderid from accounts;

** based on output from above, determine the _id value for the account to modify (1 is used below in example):

sqlite> select * from mailboxs where _account=1;       --where 1 is from _ID above
sqlite> --below command is more succinct and useful...
sqlite> select _id,_undecodename,_decodename,_shortname from mailboxs where _account=1;   --where 1 is _ID above

** the above will give a list of folder IDs and corresponding names. Hopefully, the _undecodename, _decodename and _shortname fields are equal. If not, then I’m not sure which you’d use in the step below – some trial and error might be needed, but I’d start with the _decodename values.

** if you have a large number of IMAP folders and a “typical” IMAP server, the following should list the three needed folders and their associated IDs:

sqlite> select _id,_undecodename,_decodename,_shortname from mailboxs
   ...> where _account=1 and
   ...> _shortname in ("INBOX.Drafts","INBOX.Sent","INBOX.Trash");

** Once you have the folders’ _id values you use them in the commands below, changing the _sentfolderid, _trashfolderid and _draftfolderid values to correspond with those shown in the above step. Note to change the where clause to reflect the correct account _id used above.

sqlite> --update the folder details
sqlite> update accounts set _sentfolder = 'INBOX.Sent', _sentfoldertext = 'INBOX.Sent', _sentfolderid = 23 where _id=1;
sqlite> update accounts set _trashfolder = 'INBOX.Trash', _trashfoldertext = 'INBOX.Trash', _trashfolderid = 9 where _id=1;
sqlite> update accounts set _draftfolder = 'INBOX.Drafts', _draftfoldertext = 'INBOX.Drafts', _draftfolderid = 27 where _id=1;

sqlite> --exit from sqlite
sqlite> .quit

** back at your DOS prompt, delete the files on your phone (maybe mv instead for backup purposes?? – your choice) and push the updated files

adb shell rm /data/data/com.htc.android.mail/databases/mail.db
adb shell rm /data/data/com.htc.android.mail/databases/mail.db-wal
adb shell rm /data/data/com.htc.android.mail/databases/mail.db-shm
adb push mail.db /data/data/com.htc.android.mail/databases/mail.db
adb push mail.db-wal /data/data/com.htc.android.mail/databases/mail.db-wal
adb push mail.db-shm /data/data/com.htc.android.mail/databases/mail.db-shm

**reboot the phone and hopefully all is working with the correct folders!!

adb reboot

 

Note: If your phone has sqlite3 installed on it and it is rooted, you may be able to issue the above sqlite commands without having to boot into recovery mode. I’m not sure how well the mail app would behave with it’s configuration changing while it is running.

Note2: You do this at your own risk, etc etc. It worked for me. It might work for you – or it might not.