Your best start is probably to download the quick-start rules, and the character builder demo, both of which you can find at:
http://wizards.com/dnd/TryDnD.aspx4E has a
lot of character options - the only practical way to keep track of them all is to use the character builder program. The demo has levels 1-3. The full version (levels 1-30, which is the full range of 4E levels) is only available with a subscription to the above website. Honestly, if you just plan on playing, and you are willing to spend the money, you are probably best off just buying the website subscription for the full version of character builder, and not buying any of the books. The quick start guide covers all the basic rules and terms you need.
If you want to run - I'd suggest just starting off with the quick start rules and the intro adventure, "Keep on the Shadowfell", then moving on to the Big 3: PHB1, MM1, DMG1. If you have a website subscription, the PHB and DMG are still pretty useful, but the Monster Manuals are fairly redundant to the DM Adventure Tools program, which has stats for every 4E monster from every source (books, adventures, online articles, etc.) - although the MM does have more background and "fluff" on the monsters.