When copying a large amount of data, 200GB, nominally all held in one folder but divided amongst subfolders, from one HDD to another ..........

Yorkshire_Pud · 2024-07-27T14:56:37-0400

is it better to copy individual sub folders one at a time ? Though this would be laborious

I ask because the crurrent copy is just copying the 'nominal' folder and as I am watch the progress and it seems to be copying a 'random' sequence of files.
This is making me think it is working from the 'outside' of the original HDD to its 'centre' and might mean the the copied data is similarily arranged on the recieving HDD.

GFields · 2024-07-27T15:02:09-0400

Not sure what you mean by "nominal" but most likely, if you're using Windows, go into options and make sure "view all files/folder" is selected. Then try again. Or use a migration application.

Yorkshire_Pud · 2024-07-27T15:19:20-0400

Nominal ? in my usage meant, one folder e.g. folder A, that contain all the other folder, some of which are in 'trees'.
So, folder A contains folders B to Z, and each of those folders contain other folders and each of tho.......... ad -infinitum,

I am copying "folder A' from one HDD to the other.

GFields · 2024-07-27T15:23:42-0400

Yorkshire_Pud said:
Nominal ? in my usage meant, one folder e.g. folder A, that contain all the other folder, some of which are in 'trees'.
So, folder A contains folders B to Z, and each of those folders contain other folders and each of tho.......... ad -infinitum,

I am copying "folder A' from one HDD to the other.

You can do that in Windows Explorer but as I mentioned, make sure to make the view changes. 200 GB will take a while is why I mentioned a migration application. But if this is something you only do every now and then, using Windows Explorer won't cost you anything. Data transfer is one of the services I offer with my sister company is why I have the application.

Yorkshire_Pud · 2024-07-27T15:38:07-0400

GFields said:
You can do that in Windows Explorer but as I mentioned, make sure to make the view changes.

I am,

GFields said:
200 GB will take a while is why I mentioned a migration application

It IS LOL, windows original estimate was 23 hours, but, with 2 hours elapsed, it has dropped to 2 hours with 58% remaining. It remains to be seen if that estimate is accurate lol.

djiuser12345 · 2024-07-27T15:39:34-0400

It depends on what filesystems are in use and what files you copy.

If each file (in whatever folder) is the same size 5gb the operating system might chose to copy them sequentially. Otherwise its very likely that it starts somewhere (folder A/K/E) and tries to copy 3 small files from that folder while also copying 2 big files from a different folder or at least preparing the copy for them.

A lot of variables in play here

Yorkshire_Pud · 2024-07-27T15:51:44-0400

djiuser12345 said:
It depends on what filesystems are in use and what files you copy.

The defaults win 7 file system, file sizes? from a few mb to 10s of GB.

djiuser12345 said:
Otherwise its very likely that it starts somewhere (folder A/K/E) and tries to copy 3 small files from that folder while also copying 2 big files from a different folder or at least preparing the copy for them.

That looks to be what it is doing, it has copied 18,000 files with 4,500 keft and the files currently being copied are large DJI export DATs from mavic style drones, I should have binned them, and each is taking quite a bit of time.

I am just wondering if, for 'fragmentation'-of-a-folder putposes, it would have been better to manually copy them to the second HDD folder by folder.

djiuser12345 · 2024-07-27T15:56:43-0400

Small files are problematic for speedy transfers.
Large files can be copied like a stream at full bandwidth, smaller ones create much more overhead which impacts speeds