Well, first off, for the most part Final Fantasy games are stand alone. This means that most FF games have nothing to do with the other save for the development team (sometimes) and battle system. A few exceptions are Final Fantasy X-2, which is a "remix" of sorts of FFX, but with a different battle system. If it's a sequel to a specific iteration of FF, then the original game will be in the title (ie: FFVII: Dirge of Cerberus). As for the older games - I would discourage you to avoid them, especially considering they are NOT hard to find or expensive and honestly they are much better than some of the newer ones.
I'm not going to get a popular response, but FFVII was great for its time, but it's a super easy and somewhat convoluted version of the franchise. It's a bit long, has some plot snags that relate to gameplay, and the materia system - while fun - is nowhere near as good as the job system.
If you want the older ones, I highly recommend Final Fantasy IV, V, and VI. These all do interesting things with the job system and are found on various consoles nowadays. FFIV was originally FF II on SNES when it came to our country, and FFVI is FF III on SNES in US. If you have a Wii, you can pick each of these up for $8 on virtual console. If you want to stick to the same console you own your other games on (PS3), then you can pick up the PSOne versions really cheap. On Amazon you can get the PSOne collection of FF 1&2 for $16 (and that's for a new copy, see links below) although 1 is really dated and 2 is horrible. FF Chronicles has FFIV and Chrono Trigger (an awesome RPG that features the makers of FF and Dragon Quest together) for $22.48 NEW, and FF Anthology, which has FF V and VI for $14 (best collection and deal, imo). The others are good, but don't neglect the classics, which still hold up today!
As for the new ones, FFX is getting re-released as an HD remake next year (announced at Tokyo Game Show this week) so you may want to hold off and FFXII was great. FFXI is an MMO with monthly subscription fees and hundreds of hours, I'd skip it.
In short, it's never too early to start and if you want to be an FF purist without playing games that are dated and dull, I'd recommend the FF V&VI collection, FF IX and the FF X HD remake later this year. Think about it: $14 for TWO 50+ hour RPGs and some of the best iterations of the series (FF V & VI) - do it!