Topic: How to disable duplicate files upload

(sorry for my english)
My application must disable duplicate (same file) upload. How can I do this function in plupload? please provide the soultion.

http://img413.imageshack.us/i/noduplicate.png/

Re: How to disable duplicate files upload

You could bind to FilesAdded event (Plupload Events), which gets uploader instance and reference to files array as arguments. Then cycle through the files array with your custom conditional and remove extra items with: uploader.removeFile(files[index_of_the_file_to_remove]);

If you want to see your issue fixed, do not report it here, do it on - GitHub.

Re: How to disable duplicate files upload

uploader.bind('FilesAdded', function(up, files) {
                number_files_be_uploaded = files.length;
                $.each(files, function(count, file) {
                    name_files_be_uploaded[count] = file.name;
       });
});

From above, now I bind FilesAdded event but I don't know how to gets uploader instance to check exiting with file.name arrray (name_files_be_uploaded). Could you give the guideline?

Re: How to disable duplicate files upload

If you are using queue widget and do not want to follow the pattern in the example, that I referenced above, then you can get uploader instance like this:

var uploader = $('#uploader').pluploadQueue();

where #uploader is id of your Plupload element.

If you want to see your issue fixed, do not report it here, do it on - GitHub.

Re: How to disable duplicate files upload

<script type='text/javascript'>
            $(function() {
                $('#uploader').pluploadQueue({
                    // General settings
                    runtimes : '{$runtimes}',
                    url : '{$upload_url}',
                    max_file_size : '{$max_file_size}',
                    flash_swf_url : '{$flash_swf_url}',
                    silverlight_xap_url : '{$silverlight_xap_url}'
                });
                var uploader = $('#uploader').pluploadQueue();

                var click_add_count = 0,
                     prev_obj_length = 0,
                     total_check = 0,
                     up_obj_length = 0;

                uploader.bind('FilesAdded', function(up, files) {
                    up_obj_length = up['files'].length;
                total_check = up_obj_length - prev_obj_length;
                

                    if (click_add_count >= 1){
                        var i = 0;
                        while ( i < prev_obj_length ) {
                            $.each(files, function(count, file) {
                                    if (up['files'][i].name == file.name){
                                        uploader.removeFile(files[count]);
                                    }
                            });
                            i++;
                        }
                    }
                    prev_obj_length = up_obj_length;
                    click_add_count++;
                });
            });


        </script>

Now I write the code to detect duplicate files as you suggest but I have some problem.

At first I added 01.tmp, 02.tmp, 03.tmp, 04.tmp files and then I add 04.tmp it's deleted last 04.tmp but when I add another files such as 05.tmp and 06.tmp I can't added in to the queue.

Could you suggest a logic in my javascript?

Re: How to disable duplicate files upload

Probably it will be easier to work directly on up['files'] array, as it is the one holding all the files. FilesAdded is run after files have been already added. Something like this should do:

uploader.bind('FilesAdded', function(up, files) {
        var i = 0;
        while (i++ < up.files.length) {
            var ii = i;
            while (ii < up.files.length) {
                if (up.files[i - 1].name == up.files[ii].name) {
                    uploader.removeFile(up.files[ii]);    
                } else {
                    ii++;
                }
            }
        }
    });

One should be very careful when looping through the array which is undergoing changes as the loop happens.

Actually it's quite advanced logic and I will probably integrate it into the widgets as another option.

If you want to see your issue fixed, do not report it here, do it on - GitHub.

Re: How to disable duplicate files upload

Thank you Davit, for the fix for duplicates!

Now, I'm trying to add a bit more functionality to the mix by checking for duplicates in a particular folder (or in my case photo album I'm uploading to.)

I assume I could create a pseudo "FilesAdded" array that is also used in comparison, but isn't actually factored into the new uploads?

The ultimate goal is to eventually have an option (or if no option, an alert that pops up) for the user to choose whether they want to skip duplicates or overwrite files of the same name they may have previously uploaded to an album/folder.

Perhaps, you can get me pointed in the right direction?

Re: How to disable duplicate files upload

You would need to figure out if your users want to overwrite duplicates or not and then instantiate Plupload accordingly. That will be the simplest way to do that.

If you want to see your issue fixed, do not report it here, do it on - GitHub.

Re: How to disable duplicate files upload

Thanks davit. It's better if you integrate this to be a function in pluplaod.

Re: How to disable duplicate files upload

I think I have a simple solution, it seems to work for me anyways.

// set an array of file names that are in the list
var files_in_list = new Array();

uploader.bind('FilesAdded', function(up, files) {
    $.each(files, function(i, file) {
        // check for duplicate entries
        if ( $.inArray(file.name, files_in_list) >= 0 )
        { 
            up.removeFile(file);
            return; // <-- don't process the rest of the loop for this file
        }
        // the rest is executed if file is NOT a duplicate
        files_in_list.push(file.name);
        <<code to update your ui list of selected files goes here...>>
    });

But I'm not a big javascript code master, so feel free to point out the major flaws in this technique... smile

Re: How to disable duplicate files upload

Rogier, i maked this, http://www.plupload.com/punbb/viewtopic.php?id=1242
I think that its better because your method add and after remove, in my post only dont add. If you see something wrong please tell me because im using this method.

Re: How to disable duplicate files upload

ilslabs, first of all, like I mentioned, I'm not a javascript expert.
Your code looks okay, and you've probably got it working, but I don't fully understand it.
From what I know, the 'FilesAdded' event is triggered only AFTER the files have already been added to the queue. That's why I'm removing them. I think you've found a way in your script to alter stuff right between the FilesAdded event and the QueueChanged event, right?

Anyhow, I don't think my method is overly abusing anybody's cpu, is it? wink

Re: How to disable duplicate files upload

Thanks for this subject but i need easiest way how fill easily upload.

Re: How to disable duplicate files upload

g_file_uploader.bind('BeforeUpload', function(up, file) {
            //remove duplicate file
            var upfile=_.find(up.files,function(f){return f.name==file.name});
            if(upfile&&upfile!=file)
                up.removeFile(file);
        });

_.find is underscore function .

Re: How to disable duplicate files upload

How to get the complete path and filename? Or does the browser only return this unique ID/filename for security reasons?

/test1/filename.jpg
/test2/filename.jpg

could be same, but also different. So just checking the filename is not enough.

Re: How to disable duplicate files upload

Here's another way...

I'm using the Core API so I'm taking care of the visual list of the files in the queue on my own.  So, my way of checking for duplicates also culls the short array of new files added that gets sent with the FilesAdded event, that way I can work with an accurate list of new files added.  Also, I am going under the assumption that the new files are added to the end of the main uploader.files array, so there is no need to check the new files against themselves:

uploader.bind('FilesAdded', function(up, addedFiles) {
        // remove duplicates from uploader.files and addedFiles
        var i = 0;
        while (i < addedFiles.length) {
                var dupe = false;
                var ii = 0;
                while (ii < (up.files.length - addedFiles.length)) {
                        if (addedFiles[i].name == up.files[ii].name) {
                                dupe = true;
                                up.removeFile(up.getFile(addedFiles[i].id));
                                addedFiles.splice(i, 1);
                                break;
                        }
                        ii++;
                }
                if (!dupe) { i++; }
        }
        // at this point the addedFiles array has only the non-duplicate files that
        // have been added to uploader.files, so proceed to do what we need
        // to with the new files (add them to the visual list, etc.)
});

Last edited by dylancristy (2012-07-21 22:53:00)

Re: How to disable duplicate files upload

I've been working to integrate the dylancristy's code. It does seem to work to remove the files from the internal object such that only one copy is uploaded to the server, but the jquery_ui_widget still shows both copies. Example Image

I fixed this by adding the following code to his solution:

$('#' + addedFiles[i].id).remove();

Modified Solution

    uploader.bind('FilesAdded', function(up, addedFiles) {
        // remove duplicates from uploader.files and addedFiles
        console.log("up, addedFiles", up.files,addedFiles);
        var i = 0;
        while (i < addedFiles.length) {
            var dupe = false;
            var ii = 0;
            while (ii < (up.files.length - addedFiles.length)) {
                if (addedFiles[i].name == up.files[ii].name) {
                    dupe = true;
                    up.removeFile(up.getFile(addedFiles[i].id));
/// added this line of code to remove the visual duplication from the queue
            $('#' + addedFiles[i].id).remove();

                    addedFiles.splice(i, 1);
                    break;
                }
                ii++;
            }
            if (!dupe) { i++; }
        }
        // at this point the addedFiles array has only the non-duplicate files that
        // have been added to uploader.files, so proceed to do what we need
        // to with the new files (add them to the visual list, etc.)
    });

Next step is to trigger a UI warning indicating that the duplicate was silently removed.

Last edited by filmotheklown (2013-01-22 02:34:35)

Re: How to disable duplicate files upload

i cant find the solution!