AWS EC2 file sync needs timezone set. But stat file times seem wrong

Welcome to BestSync Support Forum!
If you have any questions, comments, concerns, suggestion, please summit here, we'll try to reply you in short time.
Thank you!
Post Reply
ds00424
Posts: 17
Joined: Thu Nov 14, 2019 9:53 pm

AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by ds00424 »

Hi there.
First of all, thanks for the great tool. I have been using it for a few years (2012) now and just upgraded to 2019.

I am just starting to use AWS EC2 and I believe there is some issue with the file times after syncing.

Sync configuration:
- Folder1 is a local machine with EST timezone via FTP
- Folder2 is an AWS EC2 with UTC timezone via SFTP

When both are set to File Time: "Auto Detect", then after syncing the file times are off by a 5 hours and always need resyncing.
When Folder2 (the AWS EC2) is set to File Time: "UTC", then the syncing seems to work without showing a need to resync.

However after syncing, if i look at the file times via "stat" command or "ls -l", things don't look quite right.

For example, I started with 1 file on each side and synced both: folder1file -> folder2 and folder2file -> folder1
After the sync here are the stats of both files on both machines:
Folder1 (on local machine):
# date
Fri Nov 15 11:27:09 EST 2019
# stat folder1file --- This file did not change and so looks good.
File: `folder1file'
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: fc70b601h/4235245057d Inode: 4327686 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 505/foo) Gid: ( 505/foo)
Access: 2019-11-15 11:24:25.698526407 -0500
Modify: 2019-11-15 11:23:22.799834698 -0500
Change: 2019-11-15 11:23:22.799834698 -0500
# stat folder2file --- This file was brought from Folder2 and the Modify time is wrong.
File: `folder2file'
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: fc70b601h/4235245057d Inode: 4327677 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 505/foo) Gid: ( 505/foo)
Access: 2019-11-15 11:24:25.996539118 -0500
Modify: 2019-11-15 06:23:12.000000000 -0500
Change: 2019-11-15 11:24:26.023540269 -0500

NOTE: the modify date-time of folder2file (now on folder1 machine) is wrong
and "ls -l folder2file" shows:
-rw-r--r-- 1 foo foo 4 Nov 15 06:23 folder2file


Folder2 (on AWS EC2):
$ date
Fri Nov 15 16:28:10 UTC 2019
$ stat folder1file --- This file was brought from Folder1 and now the Access and Modify times are wrong.
File: folder1file
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 259544 Links: 1
Access: (0666/-rw-rw-rw-) Uid: ( 1001/bar) Gid: ( 1001/bar)
Access: 2019-11-15 21:23:22.000000000 +0000
Modify: 2019-11-15 21:23:22.000000000 +0000
Change: 2019-11-15 16:24:25.993619929 +0000
Birth: -
$ stat folder2file --- This file did not change and so looks good.
File: folder2file
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 259640 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1001/bar) Gid: ( 1001/bar)
Access: 2019-11-15 16:24:26.017619741 +0000
Modify: 2019-11-15 16:23:12.398195095 +0000
Change: 2019-11-15 16:23:12.398195095 +0000
Birth: -

NOTE: the Access and Modify date-times of folder1file (now on folder2 machine) are wrong.
Also now "ls -l folder1file" shows:
-rw-rw-rw- 1 bar bar 4 Nov 15 2019 folder1file
Note that there is no time show. I believe this is because it is using the Modify time and that is now in the future.
When I "touch folder1file" then the stat and ls -l look good -- as shown:
Access: 2019-11-15 16:46:57.342874461 +0000
Modify: 2019-11-15 16:46:57.342874461 +0000
Change: 2019-11-15 16:46:57.342874461 +0000


While the sync seems to work and not need repeating in this config (Folder2 set as UTC), the file times after syncing have issues.

Please let me know if I am doing something wrong, but i don't think so.

Again, thanks for the great tool and I hope this issue can be cleared up shortly.

David.
RiseFly
Site Admin
Posts: 1077
Joined: Tue Nov 03, 2009 2:51 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by RiseFly »

If the user account has not the writing privilege or do not support command to set timestamp of uploaded file, BestSync can not detect the timezone of the server, please specify the correct timezone of the server in the advanced setting of the SFTP folder browser.
ds00424
Posts: 17
Joined: Thu Nov 14, 2019 9:53 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by ds00424 »

Yes, I have the AWS EC2 set to UTC. All the output and comments above were with that setting. Sorry if I was not clear on that.
The local server is set to Auto detect.
ds00424
Posts: 17
Joined: Thu Nov 14, 2019 9:53 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by ds00424 »

A little more info...
I have the Folder1 machine set to AutoDetect. This machine is using Eastern TZ (UTC-5)
I have the Folder2 (AWS EC2) machine set to UTC. This machine is using UTC.
I create a file on both machines with same name (folder1file).
Here is the stat of each prior to any sync:
Folder1:
# stat folder1file
File: `folder1file'
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: fc70b601h/4235245057d Inode: 4327677 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 505/foo) Gid: ( 505/foo)
Access: 2019-11-16 10:28:27.854408546 -0500
Modify: 2019-11-16 10:28:27.854408546 -0500
Change: 2019-11-16 10:28:27.854408546 -0500

Folder2:
o$ stat folder1file
File: folder1file
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 259640 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1001/bar) Gid: ( 1001/bar)
Access: 2019-11-16 15:28:38.438126608 +0000
Modify: 2019-11-16 15:28:38.438126608 +0000
Change: 2019-11-16 15:28:38.438126608 +0000
Birth: -


Now when i preview the sync of Folder1->Folder2, the preview window shows:
folder1file 11/16/2019-10:28:27 > 11/16/2019-05:28:38

So the time it has determined for Folder1's file is good and matches the stat.
But the time it has determined for Folder2's file seems wrong.
Looks to be -10, when it should have been -5 -- to match UTC with ETZ.


Now with the same original files, when I set Folder2 machine to use FileTime: Auto detect, the sync preview shows:
folder1file 11/16/2019-10:28:27 > 11/16/2019-10:28:38
That looks great. But after I perform the sync, the Folder2 machine shows the following stat for folder1file:
$ stat folder1file
File: folder1file
Size: 4 Blocks: 8 IO Block: 4096 regular file
Device: ca01h/51713d Inode: 256234 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1001/bar) Gid: ( 1001/bar)
Access: 2019-11-16 20:28:27.000000000 +0000
Modify: 2019-11-16 20:28:27.000000000 +0000
Change: 2019-11-16 15:45:49.837938129 +0000
Birth: -

Access and Modify are now an extra 5 hours ahead what they should be.

And now if ask to sync again, the preview window shows:
folder1file 11/16/2019-10:28:27 > 11/16/2019-15:28:38
:-(

HTH.
ds00424
Posts: 17
Joined: Thu Nov 14, 2019 9:53 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by ds00424 »

I changed Folder2 machine to connect via FTP instead of SFTP. (I would prefer to use SFTP).
And now all time stamps and syncs seem to work as expected. Even with File Time: AutoDetect for Folder2 machine.

Not sure why SFTP vs FTP would be causing this.
Both use the same user and same folders on the machines. All I did was change the protocol for Folder2 machine

HTH.
RiseFly
Site Admin
Posts: 1077
Joined: Tue Nov 03, 2009 2:51 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by RiseFly »

I have tested the SFTP synchronization feature of BestSync, it works fine for setting/getting timestamp.
Please try to download the newest version of BestSync 2019 to see if the problem can be resolved.
ds00424
Posts: 17
Joined: Thu Nov 14, 2019 9:53 pm

Re: AWS EC2 file sync needs timezone set. But stat file times seem wrong

Post by ds00424 »

Will do.
All the above were on 14.0.1.1
I see current is 14.0.1.6 -- will try that.
Might be a few days as I am busy this week.

Thanks.
Post Reply