cdata, Coding, Exciting Techniques, Statistics, Tutorials

Operator Notation for Data Transforms

by

As of cdata version 1.0.8 cdata implements an operator notation for data transform.

The idea is simple, yet powerful.

First let’s start with some data.

d <- wrapr::build_frame(
  "id", "measure", "value" |
    1   , "AUC"    , 0.7     |
    1   , "R2"     , 0.4     |
    2   , "AUC"    , 0.8     |
    2   , "R2"     , 0.5     )

knitr::kable(d)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5

In the above data we have two measurements each for two individuals (individuals identified by the "id" column). Using cdata‘s new_record_spec() method we can capture a description of this record structure.

library("cdata")

record_spec <- new_record_spec(
  wrapr::build_frame(
    "measure", "value" |
    "AUC"    , "AUC" |
    "R2"     , "R2"  ),
  recordKeys = "id")

print(record_spec)
## $controlTable
##   measure value
## 1     AUC   AUC
## 2      R2    R2
## 
## $recordKeys
## [1] "id"
## 
## $controlTableKeys
## [1] "measure"
## 
## attr(,"class")
## [1] "cdata_record_spec"

Once we have this specification we can transform the data using operator notation.

We can collect the record blocks into rows by a "division" (or aggregation/projection) step.

knitr::kable(d)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5 
d2 <- d %//% record_spec

knitr::kable(d2)
id AUC R2
1 0.7 0.4
2 0.8 0.5

We can expand record rows into blocks by a "multiplication" (or join) step.

knitr::kable(d2)
id AUC R2
1 0.7 0.4
2 0.8 0.5 
d3 <- d2 %**% record_spec

knitr::kable(d3)
id measure value
1 AUC 0.7
1 R2 0.4
2 AUC 0.8
2 R2 0.5

And that is truly fluid data manipulation.

This article can be found in a vignette here.

Tags: