This week saw massive improvements to TXT input. I started the week with a slew of changes and as soon as I had implemented the first of them Lee Dolsen contacted me. We’ve worked together before improving PDF input. Since then he’s done a lot of work with preprocessing of PDF and other not so clean input. TXT input now auto detects the character encoding of the file. It isn’t 100% accurate but should work for the majority of cases....
I did some work with PDF output. Mainly I refactored some of the output generation code to reduce redundant sections. Over all there won’t be any user visible changes. The main reason I dove back into PDF output was because a user on OS X noted that PDF produced were not searchable. Windows users are getting searchable PDFs and on Kovid’s Gentoo Linux machine he was able to get a searchable PDF....
Like every week there were miscellaneous bug fixes. However, this week I did a bit more. TCR input and output. Do be warned that the output supports multiple compression levels; the higher levels being slower than the lower. For instance a 200K TXT file as input will take around 25 seconds on the lowest level and 3.5 minutes at the highest. TCR is an compressed text format used mainly by the Psion 3 and 5 series PDAs that were produced in the 90s....