The Art of R programming review – part 9

(This article was originally published at The Data Game - Ilan Man, and syndicated at StatsBlogs.)

It's finally come to this - the last installment of the ground breaking, epic and superlative book review. Let's jump right into it! The last three chapters of the book, which I will briefly touch upon, deal with performance enhancement, which is an important aspect of coding, and particularly in R, a common complaint of users, myself included.

Speed and memory are discussed in Chapter 14.

• There are roughly 3 ways to make R faster; (1) optimize code, (2) write certain procedures in a compiled language, (3) write code in parallel R.

This chapter deals with the optimization component - the other two come in later chapters.
• R is an interpreted language, meaning that it's going to be slower than some other programming languages, such as C/C++, which are compiled languages. A compiled language basically means that the code speaks in the same language as the machine that is executing the code. So for example, in a compiled language, the operator "+" is the same as the ADD function that the computer understands. In an interpreted language, the "+" is translated into add(x,y), which then gets read by the machine as the ADD function. So while some R code is written in C and is fast, your own code is written in R and runs slower as a result.
• As mentioned on numerous occasions, objects in R are stored in R's memory space. Specifically, R limits the memory used on any object, even on 64-bit computers, to 2^31-1 bytes. To check the size of an object in R, you can call object.size(x). Just for fun, I compared the memory space required for a matrix and data.frame of equal dimensions, as the dimensions grow up to a 1000 x 1000 matrix:

Looks about similar in size as the dimensions increase. Notice, however, how large the scale of the y-axis is. This might distort the difference for smaller dimensions. Let's see - here is the same plot, but for a matrix that's 100 rows by 100 columns:

Okay - this one starts to diverge a bit. It looks like the matrix object takes up less space than the data.frame object, but as the dimensions increase, the difference becomes smaller, as a percent of their absolute size. This means that a data.frame has more overhead than a matrix; overhead that is independent of its size. To confirm this, we would expect an even greater size difference for smaller matrices. Let's check out a 10 x 10:

Suspicions confirmed. Continuing on...

• While R loops suck, re-writing code to avoid loops won't always make it faster (such as using an apply() function), whereas vectorization as a way to avoid a loop will.
• Loops are time consuming because many aspects of the code: for (i in 1:length[x]) {} has 3 function calls! for, : and [. Recall my second post in this series. Each function call is done for each iteration of the loop, whereas in vectorizaion z<-x+y, <- and "+" is called once!
• Vector filtering is the concept that you should vectorize in R code when possible so that R runs the code in C. Examples include ifelse(), colSums(), any(), cumprod()...Remember, C is generally much faster than R!
• apply() does not speed up your functions since its written in R not C (though it does make your code clearer in my opinion). lapply() will speed up functions, however.
• The notion of vectorizing, and storing all the values in a variable at once (rather than looping) will cause R to use up more memory. This is a classic case of memory vs. speed trade off.
• Avoid using rbind() and cbind() if you can initialize a matrix or data.frame beforehand, even if its empty. Each call to cbind() is costly, whereas setting it up once (i.e. initializing an empty matrix) only costs one memory-allocation operation.
• A neat function - outer(x,y,f) - applies function f to each pair of elements of x and y.
• The author does provide examples of where using cumsum() and outer() are slower than using even a for loop.
• Vector assignment issues: remember x[2]<-7 is a call to the "[" function. Keep in mind that code with many vector re-assignment calls will slow it down more than you might imagine. This is an easy way to speed up code if you're a little extra diligent (i.e. low hanging fruit).
• Lastly, the author introduces some of R's memory packages such as RMySQL (note, I prefer using RODBC for connecting with MySQL and SQL Server databases), biglm, ff, and bigmemory.

The next chapter in the performance enhancement mini-series, is about interfacing R to other languages. I will caveat that I haven't had a chance to play around with these features much, as I'm not an experienced C programmer, but I list this here for completeness.

• Some reasons to interface with C/C++ could be for performance enhancement above and beyond vectorization, optimized code in R, and specialized I/O.
• Two ways to interface with C/C++ is .C() and .Call().
• Python aficionados will be happy to note that there is an RPy module available within Python. This allows Python users to access R within Python. However, I suspect that as Python gets more and more statistical libraries (especially pandas DataFrames), people will interface it with R less and less. It's got a burgeoning open source community and it's nature as a general, all purpose scripting tool lends itself well as a first tool to learn for those who are starting out. It's also pretty easy from a syntax perspective.

Finally, Chapter 16 discusses parallel processing with R for large computational needs. I have never worked with R in parallel, and while the author goes into some detail about parallel processing in general, and within R in particular (especially with the use of the snow package) if you are interested in this topic, I highly suggest picking up a dedicated resource to explore further. I don't think I could do this section justice by covering it, so I won't. Yup, taking the easy way out.

++++++++++++++++++++++++++++

This concludes my book review. Thanks for staying with me through this long and sometimes arduous process. I hope that this series serves as a reference point for others trying to decide if they should learn R, and if this book is the one for them. I'll reiterate that this is a different type of intro to R book, in that the author spends less time on the nuts and bolts, grammar and syntax of R, and more time explaining why R works the way it does. I would recommend this book to someone who already has a basic understanding of R and some of its functions, so that you could better understand the author's examples and code, since it can get pretty intense in certain chapters. It's worthwhile to have been coding in R for a bit of time before picking this up to better appreciate some of the tidbits of wisdom that the author drops on you.

I'm going to put the book review posts on hold for now, and focus more on real life data analysis, and the techniques used to perform said analysis. If you guys have any cool data sets you'd like to analyze, drop me a line, and maybe I can help steer you in the right direction!

Cheers.

Please comment on the article here: The Data Game - Ilan Man

Tags: books, coding, ggplot2, R programming