hubic error Unsolicited response received on idle HTTP channel

jerome     Nov 8 3:58PM 2017 GUI

I encounter this error at some point during each backup on hubic :

[17:24:28] Backup operation returned an error: 2017/11/08 17:22:13 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n

408 Request Time-out

\nYour browser didn't send a complete request in time.\n \n"; err= ERROR Failed to upload the chunk 3a379940937993b4dee8ac5ce12227d13282fc5ee3494038de738189c4a8b36b: Put https://lb1.hubic.ovh.net/v1/AUTH_0749e82d???????????????????67/default/DuplicacyBackupStorage/chunks/3a379940937993b4dee8ac5ce12227d13282fc5ee3494038de738189c4a8b36b: net/http: timeout awaiting response headers

Any idea on its origin ?


gchen    Nov 8 11:44PM 2017

This looks like their servers were trying to send back a 408 error but the response was ill-formed. The same errors have been reported here: https://github.com/ncw/rclone/issues/1583

I think the only solution is to retry on this kind of errors.


jerome    Nov 12 4:25AM 2017

Yes it's the solution to implement but not a solution I can use myself. This issue is very annoying since it makes backup impossible on hubic.

The best solution I've found is to use WIndows Task Scheduler to run a backup every minute (through a script avoiding a command window to be displayed). I use 64 upload threads but after 1-3 minutes the backup always stops. This has 2 drawbacks:

  • each time a backup restarts, it takes time, CPU and HD accesses to index the repository, making the overall backup process very slow
  • since I have a big repository to backup, uploading chunks never ends properly so no snapshot is save, and I can not recover any chunk

I won't be able to test a restore before the end of my trial period :(

Could you rapidly implement the work around you've suggested ?

EDIT :

In fact with CLI backup mostly fails due to a timeout error. You should really implement an automatic retry on connection error (3 times or more, the best would be to make it configurable) instead of stopping backup...


gchen    Nov 12 11:18AM 2017

I can implement the retry mechanism, but can you also try reducing the number of upload threads to, say, 16? The timeout may by caused by their servers not liking too many connections...


jerome    Nov 13 4:09PM 2017

Using only 16 threads doesn't make things better. It always takes a lot of time to start the backup and then skip the already uploaded chunk. Then few chunks are uploaded successfully until an error happens. It's not always the same error, sometimes a timeout, sometimes an API error, sometimes a closed connexion, ... A retry mechanism is really needed here. hubic may not be the most perfect storage, but its low cost makes it the best choice in many cases.

About the original issue of this thread, it seems that the "Unsolicited response" does not stop the backup. At least in some cases with the CLI I've seen chunks uploaded after this error displayed in the console.

When could you release a new version with a retry mechanism ? I'm ok to test a beta version if it can help you...


jerome    Nov 13 4:09PM 2017

EDIT : deletion of duplicated post


jerome    Nov 17 8:03AM 2017

Just a friendly ping...


gchen    Nov 17 11:01PM 2017

There is already a retry logic in the hubic backend. Just need to catch this timeout error. I'll come up with something next week.


jerome    Nov 19 5:43AM 2017

Great news !

Please don't focus on a specific error, in fact many different errors occur and stop the backup process: timeout, API error, closed connexion, ...


gchen    Nov 21 10:35PM 2017

I made a change to retry on any error performing the https requests: https://github.com/gilbertchen/duplicacy/commit/29bbd49a1c84761a75690f7748c780837bf43c26.

Build the CLI version from the latest source and you'll have this change. If you need a binary please let me know.


jerome    Nov 22 4:22AM 2017

It works, thanks a lot !


Log in to comment
Copyright © Acrosync LLC 2016-2017