While I have not hit your goals of 20 people in 3-5 years, I have personally built a one-man consulting firm into a 6-man profitable business in under a year starting with only $350, and I'm happy to explain how.
First of all, I started my company while working at another consulting agency and it was key to my success.
When I started I spent roughly $350 to get myself 1) legal to do business in my area under the name Full Ambit Media out of my house 2) 250 business cards from VistaPrint which were designed internally and 3) web hosting for a website designed and developed internally. From here, I continued working my job at the existing consultant firm while I worked hard (attending local industry groups, talking to friends and family) to land my first client with my new business. This allowed me to easily afford to get started without the commitments of loans or investors, sustain myself financially during the early stages and begin networking.
Shortly thereafter I landed my first client. It didn't pay as much as I had hoped, but it was feasible and it certainly helped me get the ball rolling.
Within 3 short months of starting the business I had 3 active clients, 2 of which were paying as much as I had hoped and I was extremely busy. I resigned from my position at the consulting firm and hired a bright and energetic college student from New York to work with me on development. Out of my home office with one part-time employee who I trained to follow my vision of quality we completed our first 3 projects successfully! In fact, 2 of those clients became repeat customers almost immediately thereafter and we soon had a growing list of testimonials and people being referred to us. I hired a local woman part-time to offload some of my bookkeeping and secretarial work.
Roughly 6 months after starting the business I landed a couple large contracts which demanded more development manpower and my time was growing thin with administrative and sales work. I hired a full-time general manager and an additional part-time developer to handle the demand.
About 8 months after the launch of Full Ambit Media I landed a contract with one of my existing development clients managing servers for a volume traffic website. Since that happened I have formed a wholesale relationship with a massive telecom company and hired a full-time system administrator to handle the work.
As I write this my company has been in business for 11 months and we currently employ 6 people including myself, although granted about half of them are only part-time. To this day I have never outsourced any work, I have never spent a dime on advertisement and I didn't hire any salespeople.
So, to answer your question directly, I do not think that an "off-shoring hybrid" business model is the best choice if you're truly a "domain expert."
If you want to reach your goals you're going to have to work very hard to establish your brand and reputation. I recommend that you establish a legal business name and use that for branding if that is your goal long-term. It's okay if your business is associated with you personally: My first client, who paid less than desirable and met with me at my home office for months now pays our standard rate and meets me in my private office at the Full Ambit Media headquarters. How did I manage the transition? Simple, I told him "We're growing!", smiled and got back to business. I do imagine that having a single company name, logo, website and business card from the onset through now helped the transition.
Finally, I will bestow you with the advice of my father:
Dad: "Boy, now that you're a business owner you get to choose which half of the day you want to work!"
Dad: "Yeah, the first 12 hours or the second 12 hours, take your pick."
And, honestly, that's an understatement at times.
The point is that if you're not prepared to work yourself non-stop like a dog for those 2 or 3 years then I highly suggest you start looking for a job. On the other hand, if you're prepared for sleepless nights and stressful days the reward of seeing your dreams come to fruition is completely worth the effort.
Asking for advice here on SitePoint was a great decision as it's certainly bountiful within this community. I do hope you're able to gain helpful insights from my post and I wish you the best of luck with your endeavors.