PHP – 导入CSV文件到MySQL数据库使用LOAD DATA INFILE

我有一个.csv文件数据

Date,Name,Call Type,Number,Duration,Address,PostalCode,City,State,Country,Latitude,Longitude "Sep-18-2013 01:53:45 PM","Unknown","outgoing call",'123456',"0 Secs","null","null","null","null","null",0.0,0.0,,, "Sep-18-2013 01:54:14 PM","Unknown","outgoing call",'1234567890',"0 Secs","null","null","null","null","null",0.0,0.0,,, "Sep-18-2013 01:54:37 PM","Unknown","outgoing call",'14772580369',"1 Secs","null","null","null","null","null",0.0,0.0,,, 

并使用下面的代码将数据插入到数据库中

 $sql = "LOAD DATA INFILE `detection.csv` INTO TABLE `calldetections` FIELDS TERMINATED BY '".@mysql_escape_string(","). "` OPTIONALLY ENCLOSED BY `".@mysql_escape_string("\""). "` OPTIONALLY ENCLOSED BY `".@mysql_escape_string("\'"). "` ESCAPED BY `".@mysql_escape_string("\\"). "` LINES TERMINATED BY `".",,,\\r\\n". "`IGNORE 1 LINES `" ."(`date`,`name`,`type`,`number`,`duration`,`addr`,`pin`,`city`,`state`,`country`,`lat`,`log`)"; $res = @mysql_query($con,$sql); 

但没有插入任何东西; 错误在哪里？

Answer 1

如果你会做echo($sql); 在执行之前，你会发现你的查询语法不正确，原因如下：

1. 文件名应该用引号引起来，而不是反引号，因为它是一个string，而不是一个标识符。

2. 绝对不需要调用mysql_escape_string()在FIELDS TERMINATED BY和ENCLOSED BY和ESCAPED BY子句中指定分隔符。

3. 你过分使用反斜杠 事实上，就你而言，由于没有使用保留字，所以你把它们都抛弃了。 他们只会增加混乱。

4. 在CSV文件的第一行结尾处，您必须具有 ,,,因为您将它们用作行分隔符的一部分。 如果你不这样做，你不仅会跳过第一行，而且会跳过包含数据的第二行。

5. 您不能多次使用ENCLOSED BY子句。 你必须以不同的方式处理Number字段。

6. 望着你的示例行恕我直言，你不需要ESCAPED BY 。 但是，如果你觉得你需要它像这样ESCAPED BY '\\' 。

这是一个句法正确的陈述可能看起来像这样

 LOAD DATA INFILE 'detection.csv' INTO TABLE calldetections FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY ',,,\r\n' IGNORE 1 LINES (date, name, type, number, duration, addr, pin, city, state, country, lat, log) 

---

现在恕我直言，你需要转换相当多的领域，而你加载它们：

1. 如果你的表中的datetime是datetime数据types，那么它需要被转换，否则你会得到一个错误

   不正确的date时间值：'Sep-18-2013 01:53:45 PM'列'date'在行

2. 你必须在Number域中处理单个qoutes值

3. 你很可能希望将"null"string文字更改为addr, pin, city, state, country列的实际NULL

4. 如果持续时间总是以秒为单位，那么您可以提取一个秒的整数值并将其以这种方式存储在您的表中，以便稍后可以轻松地汇总持续时间值。

这就是说声明的有用版本应该看起来像这样

 LOAD DATA INFILE 'detection.csv' INTO TABLE calldetections FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY ',,,\r\n' IGNORE 1 LINES (@date, name, type, @number, @duration, @addr, @pin, @city, @state, @country, lat, log) SET date = STR_TO_DATE(@date, '%b-%d-%Y %h:%i:%s %p'), number = TRIM(BOTH '\'' FROM @number), duration = 1 * TRIM(TRAILING 'Secs' FROM @duration), addr = NULLIF(@addr, 'null'), pin = NULLIF(@pin, 'null'), city = NULLIF(@city, 'null'), state = NULLIF(@state, 'null'), country = NULLIF(@country, 'null') 

以下是在我的机器上执行查询的结果

 mysql> LOAD DATA INFILE'/tmp/detection.csv'
     - > INTO TABLE calldetections
     - > FIELDS TERMINATED BY'，'
     - >任选地由''' 
     - > LINES TERMINATED BY',,, \ n'
     - > IGNORE 1 LINES 
     - >（@date，name，type，@number，@duration，@addr，@pin，@city，@state，@country，lat，log）
     - > SET date = STR_TO_DATE（@date，'％b-％d-％Y％h：％i：％s％p'）
     - > number = TRIM（BOTH'\'from @number），
     - > duration = 1 * TRIM（TRAILING'Secs'FROM @duration），
     - > addr = NULLIF（@addr，'null'），
     - > pin = NULLIF（@pin，'null'），
     - > city = NULLIF（@city，'null'），
     - > state = NULLIF（@state，'null'），
     - > country = NULLIF（@country，'null'）;
查询OK，3行受影响（0.00秒）
logging：3删除：0跳过：0警告：0

 mysql> select * from calldetections;
 + --------------------- + --------- + --------------- +  - ------------ + ---------- + ------ + ------ + ------ + ----- -  + --------- + ------ + ------ +
 | date| 名字| 键入| 数字| 持续时间|  addr |  pin | 城市| 状态| 国家|  lat |  log |
 + --------------------- + --------- + --------------- +  - ------------ + ---------- + ------ + ------ + ------ + ----- -  + --------- + ------ + ------ +
 |  2013-09-18 13:53:45 | 未知| 传出呼叫|  123456 |  0 |  NULL |  NULL |  NULL |  NULL |  NULL |  0.0 |  0.0 |
 |  2013-09-18 13:54:14 | 未知| 传出呼叫|  1234567890 |  0 |  NULL |  NULL |  NULL |  NULL |  NULL |  0.0 |  0.0 |
 |  2013-09-18 13:54:37 | 未知| 传出呼叫|  14772580369 |  1 |  NULL |  NULL |  NULL |  NULL |  NULL |  0.0 |  0.0 |
 + --------------------- + --------- + --------------- +  - ------------ + ---------- + ------ + ------ + ------ + ----- -  + --------- + ------ + ------ +
 3行（0.00秒）

---

最后在PHP分配一个查询string$sqlvariables应该看起来像这样

 $sql = "LOAD DATA INFILE 'detection.csv' INTO TABLE calldetections FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY ',,,\\r\\n' IGNORE 1 LINES (@date, name, type, @number, @duration, @addr, @pin, @city, @state, @country, lat, log) SET date = STR_TO_DATE(@date, '%b-%d-%Y %h:%i:%s %p'), number = TRIM(BOTH '\'' FROM @number), duration = 1 * TRIM(TRAILING 'Secs' FROM @duration), addr = NULLIF(@addr, 'null'), pin = NULLIF(@pin, 'null'), city = NULLIF(@city, 'null'), state = NULLIF(@state, 'null'), country = NULLIF(@country, 'null') "; 

Answer 2

在1分钟内在数据库中插入大于7000000条logging（超快查询与计算）

  mysqli_query($cons, ' LOAD DATA LOCAL INFILE "'.$file.'" INTO TABLE tablename FIELDS TERMINATED by \',\' LINES TERMINATED BY \'\n\' IGNORE 1 LINES (isbn10,isbn13,price,discount,free_stock,report,report_date) SET RRP = IF(discount = 0.00,price-price * 45/100,IF(discount = 0.01,price,IF(discount != 0.00,price-price * discount/100,@RRP))), RRP_nl = RRP * 1.44 + 8, RRP_bl = RRP * 1.44 + 8, ID = NULL ')or die(mysqli_error()); $affected = (int) (mysqli_affected_rows($cons))-1; $log->lwrite('Inventory.CSV to database:'. $affected.' record inserted successfully.'); 

RRP和RRP_nl和RRP_bl不在csv中，但是我们计算出来并插入之后。