Aha.. no, not a variable.  That is a folder created by some other (non s3sync) tool.  You can't sync with something else and then expect s3sync to understand it; the folder concepts of these tools are all proprietary and incompatible.

So as I suspected, your stuff was (somehow) stored to a location that s3sync doesn't see when it looks for the folder "media".  Therefore it's doing the right thing by its design and trying to store all the items over.

Best guess is the original files were sent to some differing path or something that is preventing them from being found.

You could try -d to see more details about what is being done internally.  I can't help too much more without actually having access to your source and bucket.

I just didn't ever special case the (already very "special") code to deal with single files.  s3cmd wasn't written specifically for that goal, it is just a thin command wrapper around s3.

Well if you used dry run then of course it doesn't actually sync anything so it'd *have* to start over next time wouldn't it..

But assuming you mean that the first time you didn't dry run it....
Have you considered that putting the option at the end might be a problem?  I have no idea whether that works.  Assuming your command syntax is doing what you want it to do, then transfers won't recur if the local files are the same size and md5 as the nodes out on s3.

No one ever claimed s3sync could work for single files. Try s3cmd.

I don't use the yml myself.. I just cd to the s3sync dir before issuing my commands.  Someone familiar with yml maybe could help?

None of that seems like it should be a problem.  There's no way for the code to just exit without some kind of error.. try with the -d option and maybe get some idea of what operation was pending when it "stops"?

Looks right to me.  Can you create a miniature test case of directories that fail?  If so then tar them up and email to me (contact info in README)

What percentage of s3sync users would be able to track this down by themselves in the current state?

I try to catch errors on my side and print out reasonable messages.  But I'm not going to make a deadman switch timer that checks for stalls inside ruby, that's just over the top imo.  As for file size changes and active file handling, locking is really the only way to fix it, and that assumes that the same manner of lock used is honored by all the things accessing the files.  It's a problem that's bigger than a breadbox, and there's just no way I'm going to tackle it.

Your implication is that there's some simple way to catch this and error out, but I think it's a bit of a naive assessment.  Each time between the check and the access, the answer could change.  You're never going to fix it without some kind of synchronization. 

One "right" answer, in some sense, would be to anticipate files that are likely to be open, and exclude them.

I would make no guarantees about files that change.  We don't use any kind of locking mechanism, so it's not sufficient to "re-check" the file size or something; if you are backing up things that are in motion, you need something smarter than this system!

So far, transfer stalls have never been caused by something in the s3 or s3sync code.  If you want to put the offending file up somewhere for me to pull and test with, let me know (you can find my contact info in README).

You're refering to S3SYNC_NATIVE_CHARSET? Thx for the hint, I missed that variable.

What are legal values? Is utf-8 legal? Does casing matter (ie. Windows-1252 == WINDOWS-1252)?

However, would it be possible to enhance s3sync so, that it automatically sets the charset, according to the locale?

Thx,
Alexander

Don't actually know what's legal.. I pass this to the conversion class Iconv from the ruby library.  Guess you could look there for more info.  See the file S3encoder.rb for the entirety of how I do escaping.

Also I don't know how to detect the right answer.. As I mentioned in another thread, I'm open to suggestions.

Yes.  Check the README for the native character encoding options.

Is there a reason that the official distribution is not a gem? Is help needed to make a gem as part of the official version?

I am a ruby hack and don't understand how to make gems.  Sure help would be great, either to maintain or instruct. 

I don't know of a way to guess the correct native character set, or else I would have put that in instead of making it a setting.  Anyone else is welcome to help if they know better.

(I'm not a ruby expert either!)