obsidian_sycn/编程经验/错误经验整理/TDengine.md

1. 利用Restful接口读取大量数据时报Timeout错误

py4j.protocol.Py4JJavaError: An error occurred while calling o84.load.
: java.sql.SQLException: ERROR (2318): Read timed out
	at com.taosdata.jdbc.TSDBError.createSQLException(TSDBError.java:72)
	at com.taosdata.jdbc.utils.HttpClientPoolUtil.execute(HttpClientPoolUtil.java:131)
	at com.taosdata.jdbc.rs.RestfulStatement.execute(RestfulStatement.java:66)
	at com.taosdata.jdbc.rs.RestfulStatement.executeQuery(RestfulStatement.java:37)
	at com.taosdata.jdbc.rs.RestfulPreparedStatement.executeQuery(RestfulPreparedStatement.java:45)
	at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:61)
	at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
	at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
	at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:317)
	at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.lang.Thread.run(Thread.java:748)

1. 错误原因：restful接口默认等待数据返回时间为5000ms，当数据量较大时等待时间超过5000ms导致超时；
2. 解决方法：延长等待的时间；
3. 代码：pyspark中可以用下面两种方法设置等待时间
   1. 方法一，在url中加入设置参数httpSocketTimeout=xx

df = spark.read\  
    .format("jdbc")\  
    .option("driver", "com.taosdata.jdbc.rs.RestfulDriver")\  
    .option("url", "jdbc:TAOS-RS://ip:port/db?user=user&password=password&httpSocketTimeout=xx")\  
    .option("query", "select * from test.test")\  
    .load()

	2.方法二，在option中加入设置参数httpSocketTimeout

df = spark.read\  
    .format("jdbc")\  
    .option("driver", "com.taosdata.jdbc.rs.RestfulDriver")\  
    .option("url", "jdbc:TAOS-RS://ip:port/db?user=user&password=password")\  
    .option("httpSocketTimeout", "xx") \  
    .option("query", "select * from test.test")\  
    .load()