FTP not showing all files conundrum!

Problem:

In short when querying a FTP server for a list of files, a number of files were not being shown, even though we knew they were there!

The story….

A client contacted us and asked us why we were not processing all the files they had made available on their FTP server. All files we have processed were marked with DONE in the filename. I took a look but there were no files not marked with DONE. The client insisted there were, and sent us a screenshot of said files. Oddly enough we couldn’t see these files.

After playing around with permissions, active vs passive mode, SFTP etc we weren’t a step closer to seeing the files. Trying FileZilla, Explorer, and the command line FTP didn’t show the files. BUT when I typed ‘ls’ to list files, I DID see them. Typing the command ‘ls -l’ to see attributes caused the files NOT to be shown!

Outputting the file list from ‘ls -l’ gave me and output of 817 lines (files) which totalled 70080 bytes for the listing. ‘ls’ gave me 930 lines in a total of 41198 bytes. The smaller output is because ‘ls’ doesn’t output extended attributes for each file. Now I read on some forums that FTP servers can limit the number of files returned to 2000, but obviosuly we weren’t even near this total…so that wasn’t the problem. For some reason 113 files just weren’t showing.

We contacted a collegue in Cape Town (we’re in Johannesburg). Using EXACTLY the same settings and version of FileZilla he COULD see all the files, even the ones we couldn’t when using FileZilla. So next I tried connecting to a server outside our ISA proxy server, and connecting to the FTP server using the commandline FTP. Hey Presto! I could see all files. So I thought it was a proxy issue..but after some further investigation this didn’t make sense.

Solution:

After a LOT of wasted time, I tried moving the already  processed DONE files into an archive folder, thereby decreasing the number of files in the active folder. Then suddenly the previously ‘invisible’ files were shown. So something with the number of files or size of the ouput from the directory listing is causing problems.

So while we solved the problem, I still don’t know the cause of this odd FTP behaviour 😦

Advertisements

About schuits

IT Developer for over 10 years.

Posted on September 9, 2011, in Windows and tagged , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: