On motivating:

I think having clear milestones and progress indicators would help motivate people, and make goals seem less abstract. Goals seem more tangible when you have a plan to accomplish them, and its easier to convince people that you're being realistic when you have past successes to point to.

It also helps you know how well things are going.

On scaling:

On my robotics team, scaling is easyish after a task is sufficiently split up. Basically each subtask of the greater goal has a few people (in robotics, 3-5) working together on it, and one person held accountable for the completion of that task. There are meetings in which the leader just basically joins the group to see how they're doing, and then there are other meetings where the leaders of each group convene to discuss the larger issues. For the really big changes, pretty much everyone is involved.

In robotics, splitting up the tasks are pretty easy by what skill set is needed to accomplish them. There's a programming, electronics, and drive group, along with a few groups for the various mechanical systems. Someone is in charge of coordinating all of the groups.

So basically you have 5-6 leaders, each working with a few people on a project. The 5-6 leaders work together as a team on larger decisions, and in robotics in specific there are 3 people who work together to lead the leaders, and 1 person who is entirely managerial.

Make sure that people feel included, and know that having managers is purely for simplification so that there's one point person for each project. Anyone can come up with ideas and concerns, and things within groups and the taskforce should be based on consensus.

This works pretty well for having 30ish dedicated (and 40ish nondedicated) people build a robot.

tl;dr:

Have milestones
Have a few people (enough to be held together socially) work on each project
Then, have a greater taskforce of all the project managers/leaders
Build consensus, you're working with smart people
Make sure that responsibilities are known, that way people know that if they want something to happen, they have to do it.
Responsibilities also mean that you have fewer dropped balls