tag:blogger.com,1999:blog-5467119.post5672647391645378338..comments2023-10-24T10:09:22.146+01:00Comments on The Yorkshire Ranter: Python and CSV; know your limitsAlexhttp://www.blogger.com/profile/17153530634675543954noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-5467119.post-10014114601740622152007-11-24T22:59:00.000+01:002007-11-24T22:59:00.000+01:00Another possibility: the string you're passing is ...Another possibility: the string you're passing is of the form "INSERT INTO a VALUES (['1', '2', '3', '4', '5', '6', '7'])", and the brackets might be confusing sqlite.<BR/><BR/>Try:<BR/>curse.execute('INSERT INTO airports VALUES (?,?,?,?,?,?,?)', item)Anonymoushttps://www.blogger.com/profile/04234668485051213546noreply@blogger.comtag:blogger.com,1999:blog-5467119.post-15934920984671503502007-11-24T01:29:00.000+01:002007-11-24T01:29:00.000+01:00I'm no pythonista and the docs I find with a quick...I'm no pythonista and the docs I find with a quick google look incomplete, but a few thoughts nonetheless:<BR/><BR/>1) You should probably be using SQL parameters instead of a formatted string. It will be more robust in the presence of dodgy data. Exactly what does item.str() return anyway? Is it properly quoted and escaped?<BR/><BR/>2) Does execute return an error in case of failure? If so, check it. If not, perhaps check whether the INSERT had an effect by checking conn.total_changes or something.<BR/><BR/>3) From the commit at the end (and a quick look at the docs) it appears those INSERTs are all happening in a single transaction. Perhaps that is getting too large and hitting limits somewhere. Try using autocommit mode or committing every n INSERTs.duaneghttps://www.blogger.com/profile/03807382108921381872noreply@blogger.com