What you need to remember aswell is that this is a relatively new club, and you had the first meeting. Its always very, very slow to start with, but remember, even if its just 2 of you, its still a meet, chance to get to know each other and plan for another meet, and if someone else turns up, excellent.
When 2-3 of us from the UKHCC started trying to get more people to come to our South- East meets, it was hard, but in the space of a year, 3 of us turned into nearly 10 members just meeting once a week in the evening for a coffee, and when we had our meeting on a Sunday meeting up at a pub for lunch or going to a park, there were sometimes over 20 of us.
Like I said, its slow to start with, but never give up.