Interesting, there was recently a somewhat related question posed here.
Using my experience from that question I can give a pretty large group of answers for problem 1. Pubbfr nal z, naq nffvta n ahzore bs pbvaf tvira ol n ovabzvny qvfgevohgvba jvgu a=z naq fbzr c xabja gb Obo, fb gung vg'f nf vs lbh syvccrq na vaqrcraqrag pbva sbe rnpu rairybcr.
2 is a bit tricky, since perfect mathematicians don't just eliminate the obviously wrong, they also update against the unlikely but right. Bayes' rule says that if you get a coin, you multiply your probabilities by P(got coin | X envelopes filled)/P(got coin). P(got coin) is 1/2, your chance of getting a coin if there are X coins is X/m, and your prior that there's X coins is 1/(m+1) (since 0 is a valid number of coins too). So after getting a coin, the hypothesis that there are X envelopes with coins in them gets probability 2X/m 1/(m+1).
Gut check stop. This means that for m=2, Bob would say, P(0) = 0, P(1) = 1/3, and P(2) = 2/3 after getting one coin. Looks right.
Each hypothesis leads to an expected value of (X-1)/(m-1). So we take the sum of 2X(X-1) / (m-1)m(m+1) (thanks wolfram) to get 2/3. No matter the m, the expected value for the second draw is 2/3! It's Laplace's rule of succession! Cool, huh? I'm going and giving damang an upvote just for how helpful his post was for this one. Shame about it making the problem unanswerable :P
EDIT: part two was answering the wrong question, see comment.
Each hypothesis leads to an expected value of (X-1)/(m-1)
(X-1)/m, because the emptied envelope is shuffled back into the set.
This went over well in the xkcd logic puzzle forum (my hand was not removed), so I thought I'd try it here. It came to me in a dream, so by solving it you may be helping to summon an elder god or something.
Bob replies, "That depends on what random function you used to choose how many envelopes to fill. If you, say, flipped m coins and put each one that came up heads in an envelope, the expected value is $.50."
Alice explains what her random function was, and Bob calculates the expected value. For kicks, he pays her that amount, and she lets him pick a random envelope. It has a coin in it! Bob pockets the coin. Alice then takes the now-empty envelope back, and shuffles it into the others. "Congratulations," she says. "So, what's the expected value of playing the game again, now that there's one fewer coin?"
"Same as before," Bob replies.
Problem 1: Give a value for m and a random function for which this makes sense (there are many).