when measuring voltage for 3 phase a,b,c to ground is irrelevant.
you need to measure voltages between leads eg: voltage a-b,b-c,a-c to get a proper value.
also when using capacitors the value a-b would work for your formula but in my case was actually higher with my 7.5hp idler but the ratio is a general rule60/40 value between a-b vs b-c.
eg: my final results
I have a 7.5 hp converter wired to 240v.
after ballancing I have:
B= wild leg
> A-B 255v
> B-C 255v
> A-C 238v (line voltage)
> Fitch Williams rule of thumb is line*1.08= generated legs (unloaded) so:
> 238*1.08=257v (I think I got close)
> cappacitors used:
> AB 130 uf (not counting start caps of 380 uf)
> BC 62 uf
> AC 42 uf (power factor)
my amps were: input amps on both single phase lines were 2.65A
L1- 5.45A
L2- 5.40A
L3- 3.70A (I tried hard to ballance this closer but this was the final result)
amps were measured before the transformer on the 240v side but on the output of the idler.
I used 440v run capacitors and 330v start capacitors.
these measurements were taken with a 2hp mill under light load,there is also a load from my 6kva autotransformer which upgrades my voltages to work with my 550v mill.
after connecting the L1(A) and L3(C) leg to 240v input,600 v output and the L2(B) leg to 240v input and 480 v output my volatges are 575v between each leg. right or wrong thats how I have things working right now.
as far as wiring in the capacitors goes you do need them to be a higher voltage value than what you intend to produce.
if you wire 2 or more equal voltage capacitors in parrallel you add thier capacitance values but do not effect thier voltage capacity.
if you wire 2 or more equal voltage capacitors in series you cut thier capacitance value by 1/2 but double thier voltage carring ability.
it may or may not help but here is the diagram I followed to wire my converter with a starting circuit which worked great:
http://www.wnysupply.com/index.cfm/fa/categories.main/parentcat/25431
if you use the potential relay I will have to find the box to give the model and number I used at a later date after I retrieve the box under my bench.
hope this helps you out