Pajrex: There is the whole forum topic dedicated to modifying Tevion GPS devices (sold earlier by Aldi):
GPS Australia - Tevion If you read through it, you may find what to do. I did not try it myself, but I think the principle works for many GPS devices. Most of them are running Windows CE in the background. You need to install OziExplorer CE on your SD card and get the device to run it from there. You need to have some maps on the same card so you can use it. OziExplorer is only so useful as the maps it is reading. For off road use it is not easy to find usefull maps. The best thing is if you can download a track file from someone who went there before and than just follow it.
I personally have HP Ipaq, that came with TomTom pre installed. I installed Ozi and setup is smart enough to share GPS port between two programs, so I can run Ozi and TomTom in parallel. If I switch screen to Ozi, voice navigation still works telling me where to go. But that works only on road. When you get to dirt tracks, there is no navigation. You can only look on the map and decide where to go. If you are thorough enough, you can prepare a route at home (based on map you have) and even include voice instructions.
I found that people are modifying Navman navigators to make them run OziExplorer or some other software. Sorry that I was not of much more use, but I can tell you that possibilities are there, just Google it. I would not be surprised if you find that cheapo GPS you have is the same as Aldie's Tevion, just under different name.