Open source is about publishing the code (and allowing it to be reused). You're talking about not publishing the code. Plenty of software companies don't publish binaries (e.g. Google, Facebook). Binaries or no, it's not open source if you don't even publish the code.
Nevertheless, when you have the binary, you stand a chance at reverse engineering. If you broadcast such a binary, you have a guaranteed leak. At least, when you don't publish at all, you stand a chance at actual secrecy. (Pretty unlikely, though, if too much people are involved.)
http://blakemasters.tumblr.com/post/24464587112/peter-thiels-cs183-startup-class-17-deep-thought
Some perspectives on AI risk that might be interesting. Peter is (the primary?) donor for SI, and an investor in AGI startup Vicarious Systems.