First, I would get the '1' album. It has all of their biggest hits, and is overall great.
Home is where the needle marks
Try to heal my broken heart
last.fm/user/TheRollingBones
There are two live albums. The first one (Live at the Hollywood Bowl) is very rare and is only available on vinyl. The other (BBC Sessions) can only be found online.
Anthology 1 and 3 are the best. Anthology 1 has several interesting early recordings from the fifties and early sixties when Pete Best and Stuart Sutcliffe were still in the band. Anthology 1 also has "Free as a Bird," a song Paul, George and Ringo recorded around one of John's old demo tapes after he died. Anthology 3 is justifiable because of its version of "Something" alone.
"I'm not superstitious. I'm just a little stitious."
- Michael Scott
The rooftop concert is from the movie "Let it Be." It's a documentary about the Beatles when they are recording Let it Be and Abbey Road. At the end of the movie, The Beatles play a concert on the top of 3 Seville Row in London. There is no DVD for this movie, only VHS and Laserdisc, which are both rare and out of production. You can easily find the whole movie for download on the internet. Actually, it's all on Youtube split into parts last time I checked.
The rest are either tapes of promotional videos released in the sixties or from the Anthology DVDs. All are hard to find, but you could probably purchase the Anthology DVDs from Amazon.
"I'm not superstitious. I'm just a little stitious."
- Michael Scott
This stuff has never been released, only shown on TV a long time ago. A lot of it never even shown anywhere. Basically all this video came from other fans/collectors and used to get traded around in collector trading circles. That used to be the only way to see any of this stuff, you had to know somebody.
'The Beatles at Shea Stadium' can be found on Youtube and is worth watching.
I would also spend a day watching the Beatles Anthology on YouTube. It will blow your mind if you're not a big Beatles fan and be highly informative even if you are.
Also, I wouldn't recommend purchasing a bunch of albums at once. It's easy to get overwhelmed and you won't likely listen to them all. The "1" album is a good starting point as someone suggested, but "Abbey Road" and "Help!" are also good. Also watch their movies, A Hard Day's Night and Help!. Help! is funnier.
More Beatles DLC, PLEASE!
The tracks that make the Anthology worthwhile differ from person to person. I, for example, love the version of Helter Skelter that's on Anthology, but other people might consider it a crime before God and Vishnu.
I do want to reiterate that Anthology is expensive, and unless you feel yourself completely blown away by the CDs, it's probably not a worthwhile investment. You may be better off watching the Anthology documentary. That said, Free As A Bird and Real Love should be available on YouTube (those are the new singles on the Anthology CDs)
Live at the BBC is live but it's live in a studio, not played live. But like others said, YouTube has a lot of it now, so just dig through it.
A plea for rationality on the Internet:
tinyurl.com/rbwrmc
In Giles We Trust