I lack the time to draw diagrams and so forth, so in brief, I’m going to invoke this:
Excessive pedantry tells us that it’s impossible to move in a perfectly straight line on any physical table. We all know this and yet we manage to charge, slam, and trample all the time. 
I have never once encountered a single player who correctly measured the distance “up or down” a hill, up to and including any staff I have ever played a game with. No one does that. It’s always treated as a straight-line movement even if in actuality there’s a tiny bit of trivial vertical movement involved.
We also all know that we can absolutely charge over physical terrain such as mine fields, forests, rough terrain, etc., etc. that is represented by physical three-dimensional objects. No one fails a charge when they come to the edge of a piece of physical terrain. And for that matter, I have never once seen a player properly subtract the 1/16" from their movement for moving “up” a piece of mousepad terrain.
But besides all that, Hills tell us exactly how to interact with them while moving:
You move up and down it normally. (No, “normally” isn’t defined in the rules, because no one should have to define “normally.” C’mon now. 
)
Common sense tells us that moving up or down stairs and ladders implies non-trivial vertical distance, so much so that the rules tell you you have to choose to “move up or down” them:
I’m not even going to attempt to prove that models can’t do nonsensical things like move directly up into the air when they advance, which is where examples in discussions like this usually end up. 
Conceptually and thematically, charging up a ladder doesn’t fit. (“I’m angrily charging you! Now I’m angrily climbing this 30 foot ladder at you! Hang on! …Okay now I’m angrily charging you again!”)
You get to the ladder by moving in a straight line. You move up or down the ladder. That is absolutely a non-trivial different straight line.* Models can only charge in a single straight line. citation needed**
Seeing as how we’re already at an impasse, let’s get official word on this.
Pretty please @elswickchuck , can you tell us all official-like if models can charge up ladders, stairs, or elevators? And what is the maximum duration elevator ride that is allowed? How long does a charging model have to ride in an elevator before it gets bored listening to the muzak and waiting for the elevator doors to open and forgets what it was doing there anyway?
* If the vertical distance wasn’t somehow significant, one wonders why Flight is called out as having specific interactions with obstructions and other such elevated terrain, and why Flight is written so that it essentially acts as “I absolutely do not care about vertical distance.”
** I am absolutely not going to discuss whether or not models can charge in multiple discreet straight lines, because it’s silly.
