Man, I would have to think about that too...
First, no need to burn the bridge at work, even if you do leave. If you're a good employee to them now, then they'd be happy to take you back. I've done it twice, with no regrets really.
Second, you're not buying into much of a business just based on the couple posts so far. Good that there's no debt, and good that he owns the building.
Can you keep him on, to help with existing customers' sales, and help transition the operations over to you? If so, can you two get along?
Can you go part time at your current job?
Does your wife have health insurance through her work?
If you've answered "YES" to any/all of those, then I might consider it, pending that YOU could go out and sell for new business.
I can see & sympathize with all of the 1,000 red flags already & not yet mentioned, but at 33 with a wife & 2 kids myself, I'd be thinking hard about this too...