Also coming from traditional software development, Beta usually has a less controversial definition than Alpha.
The short version is that a beta contains everything, including the bugs.
However, in games development, Beta has been taking another meaning since the last few years.
When a game enters Beta stage, it's the beginning of the end. At that point the team should only be debugging and tweaking the game. No new stuff, no additional stuff.
As for Alpha, Beta is more something defined in a contract than anything else. And contrary to Alpha, in people's minds a Beta is a phase rather than a deliverable.
In the Microsoft way of doing things, Beta is split into two major milestones: Code complete and Content Complete.
Code complete, as the name suggests, means that the programming team is done implementing every single (planned) feature. From that point on, they normally should refuse to make any modification that is not a bug fix. Ideally, being code complete would Mark the start of the beta phase. In reality, the beta phase tends to start before the game is code complete (or maybe the code complete is late, who knows? (production should)).
While the code may be complete, the game may still be missing a few assets, quests, collectables, etc. When all this done, the game is called Content Complete. This may or may not be the end of Beta.
Beta testing is obviously done during that period (provided it's actually done,which is not always the case) and should extend to after content complete, so the latest levels/missions have some time to be tested. The aim of beta testing is to find out bugs and discover potential balance issues. The games most likely to undergo beta testing are multiplayer games and/or games with high replayability.
Lately, Beta has become an en-vogue term to designate "time-limited multiplayer demos". It's a splendid way to do some data mining and balance the game (since it will generate many game sessions). However, I doubt the results are actually used by the development teams (hopefully I'm wrong).