small fraction of data is corrupted when reading files

[GregDMeyer]

I consistently seeing small chunks of data getting corrupted when I copy files off of my device. Take the following output:

$  ifuse -o ro ~/iphone/  # FYI same thing seems to happen when not using read-only
$  cp ~/iphone/DCIM/108APPLE/IMG_8280.MOV /tmp/test_1.mov
$  cp ~/iphone/DCIM/108APPLE/IMG_8280.MOV /tmp/test_2.mov
$  cp ~/iphone/DCIM/108APPLE/IMG_8280.MOV /tmp/test_3.mov
$ du -sh /tmp/*.mov
319M	/tmp/test_1.mov
319M	/tmp/test_2.mov
319M	/tmp/test_3.mov
$ sha256sum /tmp/*mov
2b75696e3d35612b62aa7ae2d198650ab2ef7f9dcf0c6ab5f8ff6832855494fd  /tmp/test_1.mov
a8622f8572dd580bb06d2fb989b38bc04404a0322432a4ebb04276322d8925be  /tmp/test_2.mov
f7fb666916cf422dd484764804ec6f15bcedc60aee03fe86837ec929f1e4a002  /tmp/test_3.mov

The files sizes are the same but the content is different! Digging in a little more using a script I wrote to compare binary files (the X's mark where each file does not agree with either of the other two):

$  python binary_visual_diff.py /tmp/*.mov
/tmp/test_1.mov: .XX........X............X..........................X.......................X..................................X...XX............
/tmp/test_2.mov: ...........................X.......................X...................X....X..........X........................................
/tmp/test_3.mov: ...X.X......XXX.............................X.....XX..................X............X........X....XXX....................X.......

So the files mostly match, but apparently some of the bytes are just wrong. I wrote some code to copy files off the devices using the "majority vote" of several copies, which seems to work, as media files can play succesfully and seem to be correct. But it is obviously not ideal.

I ran ifuse with the -d option but didn't see any debugging output (maybe I did something wrong?). Let me know how I can dig in further!

[clavisound]

Maybe some metadata are changing? iOS even re-encodes HEIF images as JPG's.

Here with latest iOS-15 are bit-by-bit same.

[nikias]

Can you check if the same thing happens when you get a file using the afcclient tool that is part of libimobiledevice tools?

[GregDMeyer]

Can you check if the same thing happens when you get a file using the afcclient tool that is part of libimobiledevice tools?

ooh, when I use afcclient I seem to get the correct file consistently, with no errors! thanks for this, at least I have a solution---let me know how I can further debug what is happening in ifuse, if it would be helpful!

Maybe some metadata are changing? iOS even re-encodes HEIF images as JPG's.

Here with latest iOS-15 are bit-by-bit same.

I was wondering this, but it seems that only a few (random-seeming) parts of the file are corrupted, and they "go back" to what they were originally on future copies. So I don't think its an issue of files changing as they are copied, unless they are being changed in a really weird/unexpected way.

---

a bit more info: I copied the file twice with afcclient and twice with ifuse, and get the following from my binary diff tool thing:

/tmp/test_afcclient_1.mov: ................................................................
/tmp/test_afcclient_2.mov: ................................................................
/tmp/test_ifuse_1.mov:     ...............................X........X.....XX..........X.....
/tmp/test_ifuse_2.mov:     ....................X...........................................

just to show that it indeed seems that afcclient is nailing it (always agreeing with the majority vote of the other files), and only ifuse is having the weird corruption happen.

[SoCuul]

I'fe noticed that for some reason it seems the same thing happens when writing files as well (over a certain size)