Lumifer comments on Selling Nonapples - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (81)
Apologies for commenting almost a decade after most of the comments here, but this is the exact same reason why "using nonlinear models is harder but more realistic".
The way we were taught math led us to believe that linear models form this space of tractable math, and nonlinear models form this somewhat larger space of mostly intractable math. This is mostly right, but the space of nonlinear models is almost infinitely larger than that of linear models. And that is the reason linear models are mathematically tractable : they form such a small space of possible models. Of course nonlinear models don't have general formulae that always work : they're just defined as what is NOT linear. In other words, linear models are severely restricted in the form they can have. When we define another subset of models suitable to the specific thing being modelled, then we will just as easily be able to come up with a set of explicit symbolic formulae. Then it will be just as "tractable" as linear models, even though it's nonlinear : simply because it has different special properties belonging to its own class of models obeying something just like the law of linearity
Not necessarily. Closed-form solutions are not guaranteed to exist for your particular subset of models and, in fact, often do not, forcing you to use numeric methods with all the associated problems.
Sorry, hadn't seen this (note to self: mail alerts).
Is this really true, even if we pick a similarly restricted set of models? I mean, consider a set of equations which can only contain products of a number of variables : like (x1)^a (x2)^b = const1 ,(x1)^d (x2)^e = const2 .
Is this nonlinear? Yes. Can it be solved easily? Of course. In fact it is easily transformable to a set of linear equations through logarithms.
That's what I'm kinda getting at : I think there is usually some transform that can convert your problem into a linear, or, in general, easy problem. Am I more correct now?
I don't think this is true. The model must reflect the underlying reality and the underlying reality just isn't reliably linear, even after transforms.
Now, historically people used to greatly prefer linear models. Why? Because they were tractable. And for something that you couldn't convert into linear, well, you just weren't able to build a good model. However in our computer age this no longer holds.
For an example consider what nowadays is called "machine learning". They are still building models, but these tend to be highly non-linear models with no viable linear transformations.