Jo, have you checked the fluid level in the transfer case? I was getting a bit of a growling noise last year in Moab, and didn't realize that my xfer case fluid was low until Dub checked it in the parking lot during the lift proceedure. Now I find out that I have a slow leak in the xfer case pan seal that I'll have to get fixed before Moab.
Also, do you have an after market 4x4 shift lever top? My VX came with a carbon fiberish looking knob from the PO and it has ALWAYS made a little "grinding" noise when I let off the gas and the gears start to push against each other. It's not really a grinding noise, but it's hard to explain.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair."
-Douglas Adams, Mostly Harmless