1. Figured bass. Figured bass uses Arabic numerals (1, 2, 3, etc.) under (or sometimes over) a given bass note to indicate which diatonic (that is, in the key) intervals are to be played above (i.e., in the right hand) that bass note. Any note you add above a bass is unaltered unless there are accidentals in the figured bass.
Many students try to reverse-engineer the figured bass symbols by trying to figure out the chord/Roman numeral first. This is a bad idea because figured bass symbols alone tell us nothing about chords or harmony. Once you add the notes above the given bass note, then try to figure out the harmony/Roman numeral. The example below highlights this. The A in the bass has a figure 6 underneath it. In the key of F major, this produces a first-inversion tonic harmony. In the key of G major, it produces a first-inversion diminished vii chord. In the key of D major, it produces a first-inversion minor iii chord. In the key of C major, it produces a major subdominant chord.
I often tell my students that figured bass evolved as a shorthand notation for species counterpoint. That is, figured bass actually suggests lines, not chords. Consider the example below:
If you look at those examples without worrying about vertical sonorities, the figured bass makes quite a lot of sense. Once you begin trying to assign Roman numerals, the task becomes a bit muddier. In the first example, we can easily understand the E-F motion in the soprano as some kind of neighbor motion or perhaps as the beginning of a passing motion. I prefer that interpretation to one which says the first chord is a root-position tonic and the second chord is a first-inversion submediant. In the second example, we see a collection of upper neighbors above a pedal bass. Sure, the notes on beat two add up to a iv chord, but is it really a chord or just a collection of neighbor notes? In the last two examples, we see suspensions notated in figured bass, complete with accidentals that tell us to raise the leading tone.
In short: figured bass tells us diatonic intervals above the bass and nothing else. If notes are to be altered, the accidentals will appear in the figured bass. Figured bass is simply a shorthand for linear motion.
2. Roman numerals. Roman numerals tell us quite a lot about a particular harmony. They tell us how the chord functions in a particular key. They can tell us quality (major chords typically get upper-case Roman numerals; minor and diminished chords typically get lower-case Roman numerals). They tell us the scale degree that forms the root of that particular chord.
Many harmony textbooks seem to favor a micro-managed approach to Roman numerals: every vertical sonority should probably have a Roman numeral. If you can't determine the Roman numeral for a given slice of music, then there's probably a non-harmonic tone stuck in there somewhere. I personally prefer a broader view, more in line with Schenker's concept of Stufen, where a Roman numeral is used to indicate the prevailing harmony of a passage. Other intermediate harmonies are accounted for in terms of passing chords, neighboring chords, etc. Such an approach draws attention to the prevailing linear organization of most music.
3. Inversion symbols. Inversion symbols are Arabic numerals affixed to Roman numerals that tell us which element of the chord is in the bass. They have a lot in common with figured bass symbols, but I think they differ in significant ways. The cadential 6-4 chord is perhaps the most glaring example of the confusion that can arise when these symbols try to play nice together. Consider the example below, which includes several different ways of indicating a cadential 6-4.
The first example suggests that the C and the E of the first chord are suspensions over a dominant harmony. The C and the E have pushed the chord tones out of the way and are creating a lovely dissonance over the bass. This example shows the linear nature of the cadential 6-4. The second example labels the first chord as a tonic-functioning harmony that just happens to have the fifth of the chord in the bass. We were probably taught somewhere along the way that this chord is consequently unstable because the fifth is in the bass. Here, the Arabic numerals attached to the chord say nothing of the linear behavior of the upper voices. The third example is a kind of fusion of the first two: it labels the individual chords as such, but the bracket underneath suggests that both chords are in the service of an overall dominant harmony. (I don't much care for this approach, since it looks too much like a secondary dominant.) The final approach again suggests two discrete entities, but highlights the instability and structural weakness of the first chord by not assigning it a Roman numeral.
A related example (and the one which sparked this whole train of thought) comes from a common cadential figure in Bach chorales:
(From chorale no. 9, "Ermuntre dich, mein schwacher Geist")
Here, the Arabic numerals are purely figured bass: they can't be confused with a chord inversion like the cadential 6-4 examples above. I think this is where my students' (and, to some extent, my) confusion began. Is there a way we can disentangle these closely related symbols without reinventing the wheel? Any thoughts?
UPDATE: We started talking about modal borrowing (a.k.a. modal mixture) today and this brought up another instance of number confusion: the bVI and bIII chords. When you borrow a mediant or submediant chord from the parallel minor and bring it into major, the root of the chord is lowered. I advise my students to interpret the Roman numeral as "in the major key, lower scale degree 3 and build a major triad on that root." The flat before the Roman numeral is an operation that applies to the root of that chord.