Use Case: I'm using python clickhouse-client to establish a connection to my clickhouse cluster and insert data. I'm copying the data from azure blob storage and my query looks something like:
INSERT INTO DB1.TABLE1
SELECT * FROM azureBlobStorage('<bolb storage path>')
SETTINGS
<some insertion settings>
The problem i'm facing is, the python client waits for the insertion to be complete and for very large tables network timeout happens (The call goes through a HAProxy and an Nginx Ingress). For security reasons i cannot increase the timeouts of the gateways.
I tried using async_insert=1, wait_for_async_insert=0 settings in the query, but I noticed it doesn't work with the python clickhouse-client.
Is there a way that upon sending an insert query from python client I immediately get the response back and the insertion happens in background at the cluster (as if i'm running a command directly at the cluster using CLI)?
async_insert=1 will not work for INSERT INTO ... SELECT ...FROM... statements, this setting have another application when you need to insert lot of small concurrent inserts
just increase network send_timeout and receive_timeout in your client