When creating a new ability how do you change it's target mechanics

Not like what it can target, but how it targets... Like

Auto-Target Self
Target Unit
Target Area

And also, specificly what exact unit can be targeted... For example... Sacrafice only can target an acolyte, but I see no indications in the sacrafice skill that sets it to only be able to target acolytes

Check on the Acolyte, perhaps?

Maybe it has an Ability that makes it "Sacrificable" and only units with that ability can be targetted by sacrifice.

I'm fairly sure that you can't alter the autocast characteristics of spells.

As far as Sacrifice is concerned, neither the Pit end nor the Acolyte end of the ability have anything that dictates which units are involved or how they interract. In fact, the targets for sacrifice would allow most units to be sacrificed, which we know doesn't happen.

I didn't even understand that...
So neither can be changed in the upgrade/abilities editor

They're hardcoded. You can't change them.

I think Acolyte Sacrifice can be changed if you change the units that have the ability, although I'm not sure on this.