Math / Trig expert help wanted :D

Lets say we have 3 points forming a triangle, as the picture suggests.

Lets say the distance between points is exactly 300.

Also, imagine we're logically going from point A, to B, to C.

I don't want to go to the exact location of B though, I want to kind of curve before I get there. (notice the red dots that illustrate what I mean).

The trick here is that the angle will be variable. The distances between points will be variable. Lets say, that at every point, the curve starts and ends 20% of the segment distance in both directions. If that doesn't make sense cause I didn't explain it right: The curve should begin on the AB segment when the distance FROM A, Towards B is equal to 80% of the length of AB. The curve would then end on the BC segment as soon as the distance FROM B, Towards C, is equal to 20% of the total distance of BC.

Usually i'm pretty good at math and when i'm not, i know where to look, but I'm not even sure where to begin looking for something like this...

If you've only got three points, you want to use a quadratic spline. For four or more, cubic splines are the way to go.
The wiki has some straightforward formulae. Note that you don't want to use X,Y as is, since that is for a 1 dimensional function. Instead, you want to interpolate two 1 dimensional functions X(t) and Y(t).

I've taken one college algebra class and I tell you that wasn't enough :)

I think I need to be using out of that wiki article is linear spline interpolation and have the curve go around the poitn B instead of inside of it. Am I thinking right?

Linear spline interpolation is just linear interpolation. It will draw straight lines between two points. What you want is to pass through the begin point and the end point such that it starts and ends parallel to the two lines. To do this use quadratic spline interpolation. I don't think quadratic spline interpolation works well for 3 or more points, so if you need to do this use cubic splines.

Bezier Curves might be easier to implement. In particular this equation:
The t E [0,1] means that at t = 0, you get the first point's coordinates, t = 1, you get the last point's coordinates out. Choose any t between to interpolate.
However, at the quadratic level I believe the bezier curve passes through all points, which is not what you intend. However, you could displace the middle point in the desired direction and fake the quadratic spline behavior.

Interesting, i'll have to play around with this. Thanks!

I've got the bezier curve function in jass if you need it

Just show it here (or send me a PM too) ;D