In the industry, that dark brown stuff is referred to as "gunk" (or mung, if it is moist). Seriously...
The aluminum is porous, and any oil will seek into the microscopic nooks and crannies and 'stain' the surface. It is generally harmless, and if it bothers you it can be 'baked off' in an oven.
The oxidation (white crust) on your aluminum bits will protect the parts. Scraping it off will just allow more of the aluminum to convert itself through corrosion. Anodizing is aluminum oxide. It's tough and protective, though in it's natural formation, ugly. Either leave it, or anodize it if you want it to be pretty.

As for the rust on your metal (steel) bits, it's natural, and may be cleaned up and sprayed with a rust converting primer. I have rust spots all over my VX due to a hack-job break in attempt (hanger marks through the paint all over the door frames, finally culminating with a screwdriver pryjob on the passenger door handle). Now that the weather's drying out, I'll be touching up these spots with said rust converting primer. Available at any auto supply store. I'll seal the job with Krylon or Undercoating or truck bed liner (depending on the location). No big deal, especially on the underside of the truck where rust prevention was never a factory concern (galvanizing is only used on body panels, and only lasts until the surface is compromised -scratched).

Sorry to go off track on the thread. I hate to see people stress out over a little bit of (unavoidable) corrosion.