按行快速vector化data.frames列表的合并
大多数关于在SO上列表中合并data.frame的问题并不完全涉及到我想要在这里得到的东西，而是随意地certificate我错了。
我有一个data.frames的列表。 我想“行”到另一个data.frame逐行。 实质上，所有第一行形成一个dataframe，第二行第二个dataframe，等等。 结果将是一个长度与我的原始data.frame（s）行数相同的列表。 到目前为止，data.frames的维度是相同的。
这里有一些数据要玩。
sample.list <- list(data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE))) 下面是我用良好的ol循环提出的。
#solution 1 my.list <- vector("list", nrow(sample.list[[1]])) for (i in 1:nrow(sample.list[[1]])) { for (j in 1:length(sample.list)) { my.list[[i]] <- rbind(my.list[[i]], sample.list[[j]][i, ]) } } #solution 2 (so far my favorite) sample.list2 <- do.call("rbind", sample.list) my.list2 <- vector("list", nrow(sample.list[[1]])) for (i in 1:nrow(sample.list[[1]])) { my.list2[[i]] <- sample.list2[seq(from = i, to = nrow(sample.list2), by = nrow(sample.list[[1]])), ] }
这可以通过使用vector化来改善，而不会有太多的脑筋？ 当然，正确的答案将包含一段代码。 “是”作为答案不算。
编辑
#solution 3 (a variant of solution 2 above) ind <- rep(1:nrow(sample.list[[1]]), times = length(sample.list)) my.list3 <- split(x = sample.list2, f = ind)
标杆
我已经使我的列表更大，每个data.frame更多的行。 我已经对以下结果进行了基准testing：
#solution 1 system.time(for (i in 1:nrow(sample.list[[1]])) { for (j in 1:length(sample.list)) { my.list[[i]] <- rbind(my.list[[i]], sample.list[[j]][i, ]) } }) user system elapsed 80.989 0.004 81.210 # solution 2 system.time(for (i in 1:nrow(sample.list[[1]])) { my.list2[[i]] <- sample.list2[seq(from = i, to = nrow(sample.list2), by = nrow(sample.list[[1]])), ] }) user system elapsed 0.957 0.160 1.126 # solution 3 system.time(split(x = sample.list2, f = ind)) user system elapsed 1.104 0.204 1.332 # solution Gabor system.time(lapply(1:nr, bind.ith.rows)) user system elapsed 0.484 0.000 0.485 # solution ncray system.time(alply(do.call("cbind",sample.list), 1, .fun=matrix, ncol=ncol(sample.list[[1]]), byrow=TRUE, dimnames=list(1:length(sample.list),names(sample.list[[1]])))) user system elapsed 11.296 0.016 11.365
尝试这个：
bind.ith.rows <- function(i) do.call(rbind, lapply(sample.list, "[", i, TRUE)) nr <- nrow(sample.list[[1]]) lapply(1:nr, bind.ith.rows)
一些解决scheme将使这个使用data.table更快
编辑 – 更大的数据集显示data.table更多。
# here are some sample data sample.list <- replicate(10000, data.frame(x = sample(1:100, 10), y = sample(1:100, 10), capt = sample(0:1, 10, replace = TRUE)), simplify = F)
Gabor的快速解决scheme：
# Solution Gabor bind.ith.rows <- function(i) do.call(rbind, lapply(sample.list, "[", i, TRUE)) nr <- nrow(sample.list[[1]]) system.time(rowbound <- lapply(1:nr, bind.ith.rows)) ## user system elapsed ## 25.87 0.01 25.92
即使使用data.frames，data.table函数rbindlist也会更快）
library(data.table) fastbind.ith.rows <- function(i) rbindlist(lapply(sample.list, "[", i, TRUE)) system.time(fastbound <- lapply(1:nr, fastbind.ith.rows)) ## user system elapsed ## 13.89 0.00 13.89
一个data.table解决scheme
这是一个使用data.tables的解决scheme – 它是类固醇的split解决scheme。
# data.table solution system.time({ # change each element of sample.list to a data.table (and data.frame) this # is done instaneously by reference invisible(lapply(sample.list, setattr, name = "class", value = c("data.table", "data.frame"))) # combine into a big data set bigdata <- rbindlist(sample.list) # add a row index column (by refere3nce) index <- as.character(seq_len(nr)) bigdata[, `:=`(rowid, index)] # set the key for binary searches setkey(bigdata, rowid) # split on this - dt_list <- lapply(index, function(i, j, x) x[i = J(i)], x = bigdata) # if you want to drop the `row id` column invisible(lapply(dt_list, function(x) set(x, j = "rowid", value = NULL))) # if you really don't want them to be data.tables run this line # invisible(lapply(dt_list, setattr,name = 'class', value = # c('data.frame'))) }) ################################ ## user system elapsed ## ## 0.08 0.00 0.08 ## ################################
data.table真棒！
警告用户与rbindlist
rbindlist速度很快，因为它不执行do.call(rbind,....)将会执行的检查。 例如，假定任何因子列与列表的第一个元素具有相同的级别。
这是我尝试与plyr，但我喜欢格洛腾迪克的方法：
library(plyr) alply(do.call("cbind",sample.list), 1, .fun=matrix, ncol=ncol(sample.list[[1]]), byrow=TRUE, dimnames=list(1:length(sample.list), names(sample.list[[1]]) ))
