用大量的testing数据填充数据库表
我需要加载一个包含大量testing数据的表格。 这将用于testing性能和缩放。
我怎样才能轻松地为我的数据库表创build100,000行的随机/垃圾数据?
你可以在GenerateData为你完成这个工作。 只需指定列名和数据types,以及如何输出数据(SQL语句,HTML,Excel,XML,CSV)。
你也可以使用存储过程 。 以下面的表格为例:
CREATE TABLE your_table (id int NOT NULL PRIMARY KEY AUTO_INCREMENT, val int);
然后你可以像这样添加一个存储过程:
DELIMITER $$ CREATE PROCEDURE prepare_data() BEGIN DECLARE i INT DEFAULT 100; WHILE i < 100000 DO INSERT INTO your_table (val) VALUES (i); SET i = i + 1; END WHILE; END$$ DELIMITER ;
当你打电话的时候,你会有10万条logging:
CALL prepare_data();
对于多行的克隆(数据复制),你可以使用
DELIMITER $$ CREATE PROCEDURE insert_test_data() BEGIN DECLARE i INT DEFAULT 1; WHILE i < 100000 DO INSERT INTO `table` (`user_id`, `page_id`, `name`, `description`, `created`) SELECT `user_id`, `page_id`, `name`, `description`, `created` FROM `table` WHERE id = 1; SET i = i + 1; END WHILE; END$$ DELIMITER ; CALL insert_test_data(); DROP PROCEDURE insert_test_data;
如果你想更多的控制数据,试试这样的(在PHP中):
<?php $conn = mysql_connect(...); $num = 100000; $sql = 'INSERT INTO `table` (`col1`, `col2`, ...) VALUES '; for ($i = 0; $i < $num; $i++) { mysql_query($sql . generate_test_values($i)); } ?>
其中函数generate_test_values会返回格式为“('val1','val2',…)”的string。 如果这需要很长时间,你可以批处理,所以你不会做这么多的数据库调用,例如:
for ($i = 0; $i < $num; $i += 10) { $values = array(); for ($j = 0; $j < 10; $j++) { $values[] = generate_test_data($i + $j); } mysql_query($sql . join(", ", $values)); }
将只运行10000个查询,每个添加10行。
我在Mihai Stancu的回复中find了很好的方法:
https://stackoverflow.com/questions/12788494/populate-a-mysql-with-a-large-series-of-rows-quickly
这里是纯math和SQL的解决scheme:
create table t1(x int primary key auto_increment); insert into t1 () values (),(),(); mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 1265 rows affected (0.01 sec) Records: 1265 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 2530 rows affected (0.02 sec) Records: 2530 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 5060 rows affected (0.03 sec) Records: 5060 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 10120 rows affected (0.05 sec) Records: 10120 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 20240 rows affected (0.12 sec) Records: 20240 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 40480 rows affected (0.17 sec) Records: 40480 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 80960 rows affected (0.31 sec) Records: 80960 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 161920 rows affected (0.57 sec) Records: 161920 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 323840 rows affected (1.13 sec) Records: 323840 Duplicates: 0 Warnings: 0 mysql> insert into t1 (x) select x + (select count(*) from t1) from t1; Query OK, 647680 rows affected (2.33 sec) Records: 647680 Duplicates: 0 Warnings: 0
尝试RandomProfile – 它们在一个前面提供了100,000条logging,非常真实的用户数据,以XML,CSV,纯文本和JSON格式导出,因此很容易导入任何数据库。