Buy Log-Linear Models
The primary reasons motivating this third edition were just to make the book more attractive on digital media and to facilitate its use with the R programming language. Beyond that, there are both large and small changes in the book. Small changes include updated figures and corrected typos.
The biggest change is the addition of two new chapters: one on exact conditional tests and the other on correspondence analysis. There is a new Subsection 1.3.1 on the controversy over using the traditional confidence intervals for binomial data. There is a new Section 1.6 on Over Dispersion and a corresponding new Section 13.5 on (Bayesian) analysis of log-linear models with random effects (over dispersion). I personally cannot see doing generalized linear mixed models as anything other than a Bayesian. They are so much easier that way. In fact, I now have a hard time justifying the idea that asymptotic inference for logistic regression would be as good as the (approximate) exact small-sample inference that can easily be obtained from modern Bayesian methods. Regardless of the form of the analysis, understanding the models is vital.
There is also a new Subsection 3.6.5 contrasting model selection using AIC with likelihood ratio testing. There is much more use of AIC in Chapter 6 on model selection. A new exercise in Section 8.3 illustrates the problems with having so many random zeros that they cause 0s in the marginal constraints associated with a log-linear model.
In the later, more theoretical, parts of the book a new Subsection 10.5.2 takes an expanded look at iterative proportional fitting and the fact that the method is not restricted to fitting ANOVA type models. That subsection also contains a new comment about a condition regarding random zeros that causes the nonexistence of maximum likelihood estimates. In Section 10.7 there previously was a question about different formulae for standardizing residuals in a log-linear model but Gupta, Nguyen, and Pardo (2007) cleared that up. A new Exercise 10.8.7 examines log-linear models subject to a linear constraint on the expected values. There is more discussion of offsets and a new Subsection 12.2.1 on them.
I have tried to include new references but I have not exhaustively brought all the references up to date. I have updated the references to my own work but I have left intact references to Christensen (1996), which is the first edition of Analysis of Variance, Design, and Regression: Applied Statistical Methods (ANREG-I), even though that book has largely been supplanted by Christensen (2015), Analysis of Variance, Design, and Regression: Linear Modeling for Unbalanced Data (ANREG-II). (ANREG-I contains much more on balanced ANOVA than does ANREG-II.) In part that was to minimize changes in the book and in part it was because a retypeset version of ANREG-I is available at http://www.stat.unm.edu/~fletcher/anreg.pdf. I did this despite the fact that Chapters 20, 21, and 22 of ANREG-II contain new material directly related to subjects in this book. In particular Chapter 20 contains more extensive discussion of some of the standard output that logistic regression programs produce than appears here.
Another key difference is that I have produced R code for this book at http://www.stat.unm.edu/~fletcher/R-LOGLIN3.pdf. The R code presupposes some basic knowledge of R and of how to specify models in R that is contained in Chapters 1 and 3 of the R code for ANREG-II which is available at http://www.stat.unm.edu/~fletcher/Rcode.pdf. Actually, (excluding the Bayesian code) most of the computing you need for this book is already available in Chapters 20, 21, and 22 of the ANREG-II code and (to a lesser extent) in the analogous ANREG-II code for SAS, http://www.stat.unm.edu/~fletcher/MinitabCode.pdf. (Yes, the SAS code is in a file named ``MinitabCode.'') I did not rerun the SAS code given in this book. (I'm not sure I have access to a newer version of SAS.) Data files for this book remain available at http://stat.unm.edu/~fletcher/llm_data.zip.
As part of the R code for the current book, http://www.stat.unm.edu/~fletcher/R-LOGLIN3.pdf contains extensive new material on modern approaches to Bayesian computing. The text itself is largely independent of the Monte Carlo methods used, but it still puts some focus on importance sampling in one section of Chapter 13. However, the computing document introduces the use of Markov chain Monte Carlo Methods.
Other than to mention the existence of the R code, I did not change the discussions of computing in the book except to eliminate many references to BMDP and GLIM. GLIM was an early program for fitting generalized linear models and has largely been supplanted by R's glm and SAS's GENMOD. Of particular note is that (according to Wikipedia) BMDP has not been available (as a stand alone product?) since 2017. Some versions of SAS prior to SAS 9.4M2 allowed you to run BMDP procedures (but I have no access to such versions), cf. https://v8doc.sas.com/sashtml/unixc/z0397594.htm. I regret that BMDP is so hard to obtain because I really like some of the options that BMDP 4F provides. People seem to have long since stopped fitting log-linear models by iterative proportional fitting, like BMDP 4F did. I don't know why the options available in BMDP 4F are not available in other programs. It may have been a computational issue (iterative proportional fitting versus iteratively reweighted least squares -- surely no longer an issue) or perhaps people writing software for generalized linear models were simply not as focused on the problems of high dimensional tables. R has capabilities for iterative proportional fitting but glm, like most programs for generalized linear models, uses iteratively reweighted least squares.
A big thank you to Fletcher Christensen, Penny Darsey, and Tanushri Srinath for helping me with the Bayesian computing and Davis Dotson for checking the general computing and getting JAGS running. Naturally, I am responsible for the final versions of everything. As previously and eternally, I owe huge debts to Ed Bedrick, Steve Fienberg, and Wes Johnson.
As the new title indicates, this second edition of Log-Linear Models has been modified to place greater emphasis on logistic regression. In addition to new material, the book has been radically rearranged. The fundamental material is contained in Chapters 1-4. Intermediate topics are presented in Chapters 5 through 8. Generalized linear models are presented in Chapter 9. The matrix approach to log-linear models and logistic regression is presented in Chapters 10-12, with Chapters 10 and 11 at the applied Ph.D. level and Chapter 12 doing theory at the Ph.D. level.The largest single addition to the book is Chapter 13 on Bayesian binomial regression. This chapter includes not only logistic regression but also probit and complementary log-log regression. With the simplicity of the Bayesian approach and the ability to do (almost) exact small sample statistical inference, I personally find it hard to justify doing traditional large sample inferences. (Another possibility is to do exact conditional inference, but that is another story.)
Naturally, I have cleaned up the minor flaws in the text that I have found. All examples, theorems, proofs, lemmas, etc. are numbered consecutively within each section with no distinctions between them, thus Example 2.3.1 will come before Proposition 2.3.2. Exercises that do not appear in a section at the end have a separate numbering scheme. Within the section in which it appears, an equation is numbered with a single value, e.g., equation (1). When reference is made to an equation that appears in a different section, the reference includes the appropriate chapter and section, e.g., equation (2.1.1).
The primary prerequisite for using this book is knowledge of analysis of variance and regression at the masters degree level. It would also be advantageous to have some prior familiarity with the analysis of two-way tables of count data. Christensen (1996a) was written with the idea of preparing people for this book and for Christensen (1996b). In addition, familiarity with masters level probability and mathematical statistics would be helpful, especially for the later chapters. Sections 9.3, 10.2, 11.6, and 12.3 use ideas of the convergence of random variables. Chapter 12 was originally the last chapter in my linear models book, so I would recommend a good course in linear models before attempting that. A good course in linear models would also help for Chapters 10 and 11.
The analysis of logistic regression and log-linear models is not possible without modern computing. While it certainly is not the goal of this book to provide training in the use of various software packages, some examples of software commands have been included. These focus primarily on SAS and BMDP, but include some GLIM (of which I am still very fond).
I would particularly like to thank Ed Bedrick for his help in preparing this edition and Ed and Wes Johnson for our collaboration in developing the material in Chapter 13. I would also like to thank Turner Ostler for providing the trauma data and his prior opinions about it.
Most of the data, and all of the larger data sets, are available from STATLIB as well as by anonymous ftp. The web address for the datasets option in STATLIB is http://www.stat.cmu.edu/datasets. The data are identified as ``christensen-llm''. To use ftp, type ftp stat.unm.edu and login as ``anonymous'', enter cd /pub/fletcher and either get llm.tar.Z for Unix machines or llm.zip for a DOS version. More information is available from the file ``readme.llm'' or at http://stat.unm.edu/~fletcher, my web homepage.
This book examines log-linear models for contingency tables. Logistic regression and logistic discrimination are treated as special cases and generalized linear models (in the GLIM sense) are also discussed. The book is designed to fill a niche between basic introductory books such as Fienberg (1980) and Everitt (1977) and advanced books such as Bishop, Fienberg and Holland (1975), Haberman (1974) and Santner and Duffy (1989). It is primarily directed at advanced Masters degree students in Statistics but it can be used at both higher and lower levels. The primary theme of the book is using previous knowledge of analysis of variance and regression to motivate and explicate the use of log-linear models. Of course, both the analogies and the distinctions between the different methods must be kept in mind.
[From the first edition, Chapters I, II, and III are about the same as the new 1, 2, and 3. Chapter IV is now Chapters 5 and 6. Chapter V is now 7, VI is 10, VII is 4 (and the sections are rearranged), VIII is 11, IX is 8, X is 9, and XV is 12.]
The book is written at several levels. A basic introductory course would take material from Chapters I, II (deemphasizing Section II.4), III, Sections IV.1 through IV.5 (eliminating the material on graphical models), Section IV.10, Chapter VII and Chapter IX. The advanced modeling material at the end of Sections VII.1, VII.2 and possibly the material in Section IX.2 should be deleted in a basic introductory course. For Masters degree students in Statistics, all the material in Chapters I through V, VII, IX, and X should be accessible. For an applied Ph. D. course or for advanced Masters students, the material in Chapters VI and VIII can be incorporated. Chapter VI recapitulates material from the first five chapters using matrix notation. Chapter VIII recapitulates Chapter VII. This material is necessary (a) to get standard errors of estimates in anything other than the saturated model, (b) to explain the Newton-Raphson (iteratively reweighted least squares) algorithm and (c) to discuss the weighted least squares approach of Grizzle, Starmer and Koch (1969). I also think that the more general approach used in these chapters provides a deeper understanding of the subject. Most of the material in Chapters VI and VIII requires no more sophistication than matrix arithmetic and being able to understand the definition of a column space. All of the material should be accessible to people who have had a course in linear models. Throughout the book, Chapter XV of Christensen (1987) is referenced for technical details. For completeness, and to allow the book to be used in nonapplied P.h. D. courses, Chapter XV has been reprinted in this volume under the same title, Chapter XV.
The prerequisites differ for the various courses described above. At a minimum, readers should have had a traditional course in statistical methods. To understand the vast majority of the book, courses in regression, analysis of variance and basic statistical theory are recommended. To fully appreciate the book, it would help to already know linear model theory.
It is difficult for me to understand but many of my acquaintance view me as quite opinionated. While I admit that I have not tried to keep my opinions to myself, I have tried to clearly acknowledge them as my opinions.
There are many people I would like to thank in connection with this work. My family: Sharon and Fletch, were supportive throughout. Jackie Damrau did an exceptional job of typing the first draft. The folks at BMDP provided me with copies of 4F, LR and 9R. MINITAB provided me with Versions 6.1 and 6.2. Dick Lund gave me a copy of MSUSTAT. All of the computations were performed with this software or GLIM. Several people made valuable comments on the manuscript; these include Rahman Azari, Larry Blackwood, Ron Schrader and Elizabeth Slate. Joe Hill introduced me to statistical applications of graph theory and convinced me of their importance and elegance. He also commented on part of the book. My editors, Steve Fienberg and Ingram Olkin, were, as always, very helpful. Like many people, I originally learned about log-linear models from Steve's book. Two people deserve special mention for how much they contributed to this effort. I would not be the author of this book were it not for the amount of support provided in its development by Ed Bedrick and Wes Johnson. Wes provided much of the data used in the examples. I suppose that I should also thank the legislature of the state of Montana. It was their penury that motivated me to begin the project in the spring of 1987. If you don't like the book, blame them!