= 27d567bc2d48d9f40e9ccc20ea370b15)
trist comments on The Power of Noise - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (80)
Well, I don't know of a single piece of software which requires that its inputs come from specific probability distributions in addition to satisfying some set of properties.
In fact, wouldn't that software object have to maintain some running counts to even be able to estimate from which distribution do its inputs come?
This is in some sense my point. If you want to be so hardcore Bayesian that you even look down on randomized algorithms in software, then presumably the alternative is to form a subjective probability distribution over the inputs to your algorithm (or perhaps there is some other obvious alternative that I'm missing). But I don't know of many pieces of code that require their inputs to conform to such a subjective probability distribution; rather, the code should work for ALL inputs (i.e. do a worst-case rather than average-case analysis, which will in some cases call for the use of randomness). I take this to be evidence that the "never use randomness" view would call for software that most engineers would consider poorly-designed, and as such is an unproductive view from the perspective of designing good software.
Any software that uses randomness requires you to meet a probability distribution over its inputs, namely that the random input needs to be random. I assume that you're not claiming that this breaks modularity, as you advocate the use of randomness in algorithms. Why?
Based on the discusssions with you and Lumifer, I updated the original text of that section substantially. Let me know if it's clearer now what I mean.
EDIT: Also, thanks for the feedback so far!
The probability distribution part is better, though I still don't see how software that uses randomness doesn't fall under that (likewise: compression, image recognition, signal processing, and decision making algorithms).
If the software generates its own randomness (for instance, in JAVA you can do this by creating a new instance of a Random object and calling nextInt(), etc.) then I don't see how this breaks modularity.
Compression algorithms like bzip don't promise to make things smaller as part of their contract, they only promise to be lossless. To the extent that a user relies on them becoming smaller consistently, this can lead to trouble, for instance if I expect by inputs of size 10 kilobytes to compress down to 2 kilobytes, and then many of the inputs in a row stay at 10 kilobytes, I can get unexpected load that could create issues.
I don't know of many image recognition algorithms that are integrated into a large system without a human in the loop, except perhaps Google Image Search which arguably has a human (the end-user) that filters the results at the end. This is precisely due to their non-modularity: they fail in unexpected and difficult-to-characterize ways, such that any attempt to enforce a contract or abstraction would be probably misguided at this point. The best they can currently promise is "we correctly implement the fast Fourier transform" and leave it up to the programmer to decide whether an FFT is what is merited in a given case.
ETA: Another way to put the bzip example which might fit more with your language, is that yes the guarantee of bzip is that it is lossless and that it will make your data smaller as long as the data fit the properties that bzip attempts to exploit, and if that is what we want the contract to be then I would agree that bzip is non-modular.
nitpick: Java default PRNG is a linear congruential generator. It's not as bad as the infamous RANDU, but I won't suggest to use it for anything that needs more than a hundred or so pseudo-random bits.
bzip is a great source of random bits.
:)