runtime error: invalid memory address or nil pointer dereference

nzevth    Nov 15 3:16PM 2016 CLI

I've noticed this fatal crash a few times now during backup. It outputs the following when it occurs:

Uploaded chunk 1220 size 21316196, 1.40MB/s 04:42:55 50.0%

runtime error: invalid memory address or nil pointer dereference

goroutine 9432 [running]: runtime/debug.Stack(0x0, 0x0, 0x0)

    /usr/local/go/src/runtime/debug/stack.go:24 +0x80

runtime/debug.PrintStack()

    /usr/local/go/src/runtime/debug/stack.go:16 +0x18

github.com/gilbertchen/duplicacy.CatchLogException()

    /Users/chgang/zincbox/go/src/github.com/gilbertchen/duplicacy/duplicacy_log.go:166 +0x20f

panic(0xa7ad60, 0xc82000e0b0)

    /usr/local/go/src/runtime/panic.go:426 +0x4e9

google.golang.org/api/gensupport.(*ResumableUpload).transferChunks(0xc82161a3c0, 0x7fba06cd65d0, 0xc82000ef50, 0xc8202ca940, 0x0, 0x0)

    /Users/chgang/zincbox/go/src/google.golang.org/api/gensupport/resumable.go:89 +0x617

google.golang.org/api/gensupport.(*ResumableUpload).Upload(0xc82161a3c0, 0x7fba06cd65d0, 0xc82000ef50, 0xc8200e2820, 0x0, 0x0)

    /Users/chgang/zincbox/go/src/google.golang.org/api/gensupport/resumable.go:114 +0x5e

google.golang.org/api/drive/v3.(*FilesCreateCall).Do(0xc831eb19e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)

    /Users/chgang/zincbox/go/src/google.golang.org/api/drive/v3/drive-gen.go:3083 +0x54c

github.com/gilbertchen/duplicacy.(*GCDStorage).UploadFile(0xc8200e8840, 0xc8203ca5a0, 0x47, 0xc82c120000, 0x118bab8, 0x403c12a, 0x0, 0x0)

    /Users/chgang/zincbox/go/src/github.com/gilbertchen/duplicacy/duplicacy_gcdstorage.go:552 +0x616

github.com/gilbertchen/duplicacy.(*BackupManager).UploadChunk.func1(0xc8201d3ff0, 0xc82013e0c0, 0xc8388a51c0, 0xc820132000, 0xc8203ca5a0, 0x47, 0xc8388a5180, 0x40)

    /Users/chgang/zincbox/go/src/github.com/gilbertchen/duplicacy/duplicacy_backupmanager.go:1743 +0xe4

created by github.com/gilbertchen/duplicacy.(*BackupManager).UploadChunk

    /Users/chgang/zincbox/go/src/github.com/gilbertchen/duplicacy/duplicacy_backupmanager.go:1752 +0xef9

I've enabled debug logging to see if I can catch any more info if/when it happens again. I'm using Google Drive as a backend on Ubuntu 16.04 x64.


gchen    Nov 15 7:50PM 2016

This is a bug in the official Google Drive client -- it doesn't check if the response is nil before reading the http status code. I'll try to get a fix very soon.


gchen    Nov 15 11:55PM 2016

A new version 1.1.6 is now available. For this version I upgraded the Google Drive client library as well as the Go compiler (1.6 -> 1.7.3). Let me know if this problem persists.


nzevth    Nov 16 2:54AM 2016

Thanks for the speedy fix, however it's just thrown another error while skipping over previous chunks:

Skipped chunk 22690 size 6041048, 17.51MB/s 02:52:20 37.6% Failed to find the path for the chunk 5dac616994c43928ce987a463d0dbef2434b8940d4f82cfbba271d451dd6fa21: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files%28name%2C+mimeType%2C+id%2C+size%29&q=name+%3D+%275dac616994c43928ce987a463d0dbef2434b8940d4f82cfbba271d451dd6fa21%27+and+%270B2ke_ro3jI2zUW1Va1dQcE5hLUk%27+in+parents: http2: server sent GOAWAY and closed the connection; LastStreamID=9741, ErrCode=NO_ERROR, debug="max_age"

The backup itself then failed.

My first thought is potential throttling - does the Google Drive library handle throttle requests/backoff, or would it just pass the failed request back to Duplicacy itself, which then aborts?


gchen    Nov 16 1:56PM 2016

I uploaded a new version that should handle this GOWAY error. The version number is still 1.1.6.


nzevth    Nov 16 4:41PM 2016

Was the Linux x64 1.1.6 build updated as well? I just got the error again with duplicacy_linux_x64_1.1.6 - SHA1 hash is 21D976DE0C2D4D3BF1A6E0A0E8A9A3F35FE57E1D. This time it was while uploading rather than skipping previously uploaded chunks.

Uploaded chunk 339 size 22029391, 1.74MB/s 1 day 06:19:15 3.1% Failed to upload the chunk 60d900a914be3e3070c0c52c0bb0569e1cf6caf591642d5f5cbd3c9056fb5d84: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id&uploadType=resumable&upload_id=......: http2: server sent GOAWAY and closed the connection; LastStreamID=3089, ErrCode=NO_ERROR, debug="max_age"

I cut out the upload_id as I don't know if its sensitive or not.


gchen    Nov 16 10:07PM 2016

There is a bug in previous fix and as a result the GoAway error is still no handled. The new build (still 1.1.6) can be downloaded from our github page (sha1: 6cbc32a15a60b21ecebd6ba0d76af4cfa6a43784).


nzevth    Nov 17 4:42AM 2016

I just got the error again using duplicacy_linux_x64_1.1.6 - sha1 6cbc32a15a60b21ecebd6ba0d76af4cfa6a43784.

Failed to upload the chunk f29d0493f639a1c38f716f6b030cf262987cf1a59d60d6b1af816bd5d860dfb0: Post https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id&uploadType=resumable&upload_id=......: http2: server sent GOAWAY and closed the connection; LastStreamID=3005, ErrCode=NO_ERROR, debug="max_age"


gchen    Nov 17 11:35PM 2016

I finally figured out the error type when an http2 GOAWAY message is received -- it is http2.GoAwayError, but wrapped in url.Error. That is why previous attempts to catch http2.GoAwayError failed.

The sha1 of the new linux binary is 2de54c72a6f8103cad81da8cf319ba70acd60709.

Sorry for not getting this right the first two tries.


nzevth    Nov 18 12:28AM 2016

No worries. I'll put it through its paces overnight and hopefully it completes this time.


Log in to comment
Copyright © Acrosync LLC 2016