This is known as compounding interest. In my more technical blog I derived the interest paid to that account, see it here. It is a differential equation whose solution is the exponential function
Where k is the interest given by the bank.
Could everybody be infinitely rich? Of course not. For these interests to be payable either there is debasement of the currency or the market should be flooded with products manufactured at the expense of this planet's billion-year energy reserves (be more aware of what surrounds you, you can download for free the excellent manuscript by D. MacKay). Until today we have used the second option. If we run out of energy, the first option will come suddenly.
Also read the interesting post at Do the Math displaying the views of an economist and a physicist about the topic. Some excerpt for your delight:
Physicist: Before we tackle that, we’re too close to an astounding point for me to leave it unspoken. At that 2.3% growth rate, we would be using energy at a rate corresponding to the total solar input striking Earth in a little over 400 years. We would consume something comparable to the entire sun in 1400 years from now. By 2500 years, we would use energy at the rate of the entire Milky Way galaxy—100 billion stars! I think you can see the absurdity of continued energy growth. 2500 years is not that long, from a historical perspective. We know what we were doing 2500 years ago. I think I know what we’re not going to be doing 2500 years hence.