Exepack-Decompression and TTArchive Support



Exepack-Compression is a compression algorithm which was initially developed to compress executables (-pack option of TIGCC compiler) and to generate eBooks for TICT's eBookReader. Everything (tools, sources, headerfiles etc.) which is necessary to use this compression in your own projects can be found in the TIGCC Tools Suite which can be download from the TICT-HQ at http://tict.ticalc.org.

To make things easier for you I have decided to integrate support for exepack decompression and ttarchives into library ExtGraph. It's no longer necessary nor recommended to include files ttunpack.h and ttarchive.h into your sources. Just include extgraph.h and link against extgraph.a. That's all. To generate the compressed data files and ttarchive files you still need the TIGCC Tools Suite, of course (especially the tools ttpack and ttarchive).



What is the difference of an exepacked file and an ttarchive?

An exepacked file is just a single data file which is compressed using tool ttpack. TTArchive files are, as the name already implies, archives which can hold many entries not just one. Each entry can be compressed using ttpack or any other program before you stuff it into the TTArchive file. Tool ttarchive won't care about the content of the entries at all.



How to get the Pointer to the Beginning of a File?

To use the exepack decompression routine and/or the ttarchive access macros you'll need a pointer to the memory where the external variable is stored first. This can be simply done function like the following:


//------------------------------------------------------------------
// returns pointer to start of external variable which name is given
// by fname.
// if variable cannot be found NULL is returned
//
// NOTE: this routine internally locks the variable. Due to the fact
//       that the variable has to be unlocked again if you don't
//       need it anymore, it uses the given pointer to a HANDLE
//       to store the handle of the locked variable in it.
//------------------------------------------------------------------
unsigned char* GetPointerToFile(char* fname,HANDLE* h) {
    SYM_ENTRY*     symptr;
    unsigned char* src;

    if ((symptr = DerefSym(SymFind(SYMSTR(fname))))) {
        *h = symptr->handle;
        if (!(src = (unsigned char*)HLock(*h))) return NULL;
            src+=2;
            return src;
        }
    else {
       return NULL;
    }
}

How to use the ExePack Decompression Routine?

The Exepack decompression routine needs takes two pointers. A source and a destination pointer. The source pointer points to the memory which holds the exepacked destination and the destination pointer should point to a buffer which is large enough to hold the decompressed data. Suppose you have an external variable which hold the compressed data (generated with tool ttpack) and you have already implemented the above function GetPointerToFile(). Then decompressing this file can be easily done with a function like this:


//------------------------------------------------------------------
// decompresses an external variable and returns an allocated buffer
// (don't forget to free the returned buffer)
//
// if an error occurs this function returns NULL
//------------------------------------------------------------------
unsigned char* GetDecompressedData(char* fname) {
    HANDLE         h;
    unsigned char* src = GetPointerToFile(fname,&h);
    unsigned char* dest;

    if (!src) return NULL; // file not found

    // check if data is really exepacked
    if (!ttunpack_valid(src)) {
        HeapUnlock(h);  // data NOT valid - unlock locked variable again!!
        return NULL;
    }

    // allocate buffer for decompressed data
    if (!(dest = malloc(ttunpack_size(src)))) {
        HeapUnlock(h);  // out of memory - unlock locked variable again!!
        return NULL;
    }

    // decompress the data
    if (ttunpack_decompress(src,dest) != TTUNPACK_OKAY) {
        free(dest);
        dest = NULL;
    }

    HeapUnlock(h);
    return dest;
}

How to get an Entry of an TTArchive file?

The first step to retrieve an entry of an archive is to get again a pointer to the start of the external variable which holds the ttarchive. The following code shows how to loop through all entries of an ttarchive which is stored in a variabled called "mydata". Of course, the following loop is not really useful, but it demonstrates almost everything which is necessary to now about TTArchives.

HANDLE         h;
unsigned char* src = GetPointerToFile("mydata",&h);

if (!src) {
    // variable not found !!!
}
else {
   if (ttarchive_valid(src)) {
       unsigned short i;
       unsigned short nr_entries = ttarchive_entries(src); // get number of entries

       for (i=0;i<nr_entries;i++) {
           unsigned char* pointer_to_entry = ttarchive_data(src,i);
           unsigned short size_of_entry    = ttarchive_desc(src,i)->length;
           unsigned char* decompressed;

           //-----------------------------------------------------------------------
           // Now we have a pointer to the entry and we know how large this entry is
           // If the entry is compressed we could easily decompress it like this:
           //-----------------------------------------------------------------------
           if (ttunpack_valid(pointer_to_entry)) {
               // this entry is compressed ...
               if ((decompressed = malloc(ttunpack_size(pointer_to_entry)))) {
                   if (ttunpack_decompress(pointer_to_entry,decompressed) == TTUNPACK_OKAY) {
                        //---------------------------------
                        // entry successfully decompressed.
                        // do something with it.
                        //---------------------------------
                   }
                   free(decompressed); // free allocated memory for decompressed entry again
               }
               else {
                   // out of memory!!
               }
           }

           else {
               // this entry is NOT compressed. Do something else with it ....
           }
       }
   }

   HeapUnlock(h); // unlock variable again ....
}
[The above routines are apriori tested thanks to Malcolm Smith (TRgenius). They used to contain at least one severe bug.]

IMPORTANT NOTE: If your program already uses library ExtGraph in combination with ttunpack.h and/or ttarchive.h from the TIGCC Tools Suite, please REMOVE all #include "ttunpack.h" and #include "ttarchive.h" lines in your sourcecode. Including one of these files is not longer necessary NOR recommended, all the more ExtGraph contains newer, faster and smaller versions. Just include extgraph.h, that's enough.

BTW: Is anyone out there who may extend the docs about Exepack and TTArchive to become more useful? A short demo, which shows the usage of the decompression and the ttarchive stuff would be nice, too.

If "yes", please send me a mail at thomas.nussbaumer@gmx.net.