Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

DanielVarga comments on Self-fulfilling correlations - Less Wrong

103 Post author: PhilGoetz 26 August 2010 09:07PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread. Show more comments above.

Comment author: DanielVarga 28 August 2010 09:56:04PM 2 points [-]

[...] that seems to make vanilla SA incredibly non-useful in the modern era of parallel computing.

That is what parallel tempering is for.

Comment author: [deleted] 14 February 2012 01:19:00AM 0 points [-]

Also, SA is specifically useful when the original objective function can be easily evaluated, but its derivatives are too expensive. With SA, you don't need to compute derivatives or the normalizing constants. You can try quasi-Newton methods and other approaches, but even these are computationally intractable in many cases. There are certain ways in which a problem can be non-convex that makes SA an attractive alternative. In principle, this could be true even in low dimensional problems, meaning that it's not at all just a question of parallelism. Another thing worth mentioning is that SA lends itself very well to the GPU in some cases where traditional optimizers don't.