In regression and ANOVA, an interaction occurs when the effect of one independent variable on the dependent variable is different at different levels of another independent variable. When one or both of the independent variables is categorical, then two common strategies for dealing with interactions are stratifying and adding an interaction term. A somewhat less common method is classification and regression trees. Each has its advantages and disadvantages.
If you often have regression problems in which you have a great many independent variables, partial least squares is a technique you should know about.
The chi-square test can refer to several different types of tests. Here I will discuss the one-way and two-way tests. The two-way test generalizes to multi-way tests in a natural way. These tests are tests for nominal variables (for a discussion of what a nominal variable is, see this post). The one-way test tests whether a variable is distributed according to some proportions that you specify beforehand. The two-way and multi-way tests test whether two (or more) variables are associated.
The title of this post is a quote from baseball great Yogi Berra. Yogi was famous for saying things that sounded kind of strange, but had some wisdom in them, but I bet he never thought he’d be quoted in a statistics blog!