• Registered: 2011-04-29
  • Posts: 3

Re: Large files > 2GB

Hey guys,

found your project and really like it!

I was just wondering if anybody could tell me when the next version will be released and if there is any fix for the large file (>200mb) issues. The only runtime I can get to work reliably on most browsers is html4.
(exept gears is working perfect, but only on chrome)

I was wondering if jprogrammer could upload the fixed xap again or someone could send it to me.

Guess there are quite some people interested in the project, but files >200mb are not unusual nowadays.

Thanks for your help and for all the time you put in the project!

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@Northwood that xap has been included in the latest release. What kind of problems you got with files >200mb. It should go pretty smooth all the way to 1gb.

If you want to see your issue fixed, do not report it here, do it on - GitHub.

53 Reply by Northwood (edited by Northwood 2011-05-02 09:57:12)

  • Registered: 2011-04-29
  • Posts: 3

Re: Large files > 2GB

Hey davit,

I was testing plupload on a clean xampp installation.
The problem is that I have only 2GB of memory (mostly around 1GB free), but for my application the system should at best also work for a bit older computers. smile

I played around a lot with the different runtimes and came to following results:
Flash: sometimes works, sometimes not. Without chunking I got it running up to 1.9gb. But with chunking more problems. (just saw that it takes around 2gb virtual memory for a 500mb file with chunking and stoped at 26%)

Silverlight: Same problem as flash. Files >500mb sometimes stop at around 50%. Last test the plugin crashed (and taking around 2gb of memory)

html5
max working test file 50MB. Firefox chews up already 1gb memory. Larger files than and upload doesn't start at all

Same problems in MSIE.

Only runtime working always nicely is gears. Maybe I should limit usage to Chrome smile
(and of course HTML4 is running nicely)

May it be that it's manly a memory problem? But why would a 500mb file take 2gb on a chunked upload?

How is it going with the java runtime? I understand that it all takes time and really appreciate all the work you put into it! Just curious...

  • b007
  • Member
  • Offline
  • Registered: 2011-05-02
  • Posts: 26

Re: Large files > 2GB

So all Files Until 2GB works fine?
in any runtime ?

  • Registered: 2011-04-29
  • Posts: 3

Re: Large files > 2GB

Again in simpler words: smile

Silverlight: max ~200mb (sometimes more, mostly not)
HTML5: max ~100mb
Flash (with chunking): max ~200mb (sometimes more)
Flash (without chunking): >2GB works _mostly_ fine
Gears: >2GB works fine
HTML4: max 2GB

It is a bit browser dependent, but these are only approximate values.
I can get it to work, but only with cut backs (no chunking or HTML4)

As I said, maybe a memory problem, I will try it on a machine with more memory and post the results later..

The interesting thing i found is that HTML4 is actually not loading the whole file into memory as it was said here... (HTML5 may do that and thats why its not working?!)

  • b007
  • Member
  • Offline
  • Registered: 2011-05-02
  • Posts: 26

Re: Large files > 2GB

So what is the best way?

Just limit clients to upload 2GB max files?

and use flash and use it without chunking?

Thanks..

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@Northwood, it's quite strange that SilverLight in your case also requires lot of memory, maybe you were trying it just after another runtime, with maybe clogged memory after last hang? SilverLight reads files in streams, so if it hangs it should be some other reason.

Java won't come out any time soon, however it is definitely scheduled for Plupload 2.

If you want to see your issue fixed, do not report it here, do it on - GitHub.
  • Registered: 2011-05-12
  • Posts: 1

Re: Large files > 2GB

Here is an interesting reading about various runtime/browser file size limitations from java runtime for plupload developer http://www.cabo.dk/blog/jakobs-blog/survey-of-large-browser-uploads

  • Registered: 2011-03-02
  • Posts: 3

Re: Large files > 2GB

Soroush wrote:

Here is an interesting reading about various runtime/browser file size limitations from java runtime for plupload developer http://www.cabo.dk/blog/jakobs-blog/sur … er-uploads

It is a little confusing, he doesn't really explain what he means by streaming.  I assume it means 1) read a chunk from disk 2) send it to the server 3) repeat steps 1 and 2 until EOF.   This would be different than 'chunking' which is 1) Read the whole file into browser memory 2) chunk it up 3) send a chunk to the server 4) repeat step 3 until you're out of chunks. 

If I'm understanding correctly then all other solutions other than java applet will always be limited.  Java applets also let you upload whole folders.  jakob lists the java limitations (long pre-load, js to java communication) and I would like to add one.  They are all quite ugly compared to Flash,HTML UIs.

60 Reply by MSP

  • MSP
  • New member
  • Offline
  • Registered: 2011-05-12
  • Posts: 1

Re: Large files > 2GB

So flash w/o chunking is the best to upload greater than 2GB?  I have Flash Builder 4 with Flex, does anyone have an example of how this is implemented in Flex or any documentation to go along with the plugin?

61 Reply by sasha_k (edited by sasha_k 2011-06-06 10:37:49)

  • Registered: 2011-06-06
  • Posts: 3

Re: Large files > 2GB

Hi,

Is updated Silverlight application included into 1.4.3.2 release?
If no where new build of Silverlight with fix for 2gb can be downloaded?

Silverlight application from 1.4.3.2 generates IO_ERROR in all browsers on my PC for files greater than 700mb.
Looks like Silverlight doesn't initiate connect to server in those cases, because I don't see any records in Apache's access log.

But I see records in Apache's access log for files with size less than 100mb and all works without IO_ERRORs in that case.

Best regards, Alexander.

  • Registered: 2011-06-06
  • Posts: 3

Re: Large files > 2GB

Does Silverlight work with big files good using chunking mode only?

  • Registered: 2011-06-12
  • Posts: 1

Re: Large files > 2GB

does silverlight works with gif format too???

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@sasha_k, yes the fix was included, but it wasn't stable for me, that's why we didn't want to merge it for so long, but then there was a high demand for this. Unfortunately current structure of the framework doesn't allow a fully reliable upload for big files. We can't add it to current version, since it will require a major rewrite.

Proper support will be coming as Plupload 2.

If you want to see your issue fixed, do not report it here, do it on - GitHub.
  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@dj_jd2x all runtimes work with gif files, what makes you wonder?

If you want to see your issue fixed, do not report it here, do it on - GitHub.
  • Registered: 2011-06-06
  • Posts: 3

Re: Large files > 2GB

@davit, thank's for letting know about plans for this feature. Looking forward to getting Plupload 2.
Our team has found way how to extend your Silverlight application for our needs to work with big files now.
We've used Client HTTP stack instead of Browser HTTP (http://msdn.microsoft.com/en-us/library … s.95).aspx). We've imported cookies from browser to be used with Client HTTP stack.
Also we've compiled in Silverlight 4 mode to allow disabling write stream buffer.
Now it works for our needs.
Hope this information can help your team.

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@sasha_k, thanks for the information. How big are the files you upload without problems with this? Are there any fails at all?

If you want to see your issue fixed, do not report it here, do it on - GitHub.
  • Registered: 2011-06-04
  • Posts: 3

Re: Large files > 2GB

2 GB files should be no problem at all i think.

  • Registered: 2010-08-13
  • Posts: 3

Re: Large files > 2GB

Hi guys,

An example using the plupload java runtime is running at: http://www.jdams.org/plupload

And for a survery of large uploads see: http://www.cabo.dk/blog/jakobs-blog/sur … er-uploads

kind regards,
Jakob

70 Reply by Eric Zimmerman (edited by Eric Zimmerman 2011-07-26 18:33:50)

  • Registered: 2011-05-11
  • Posts: 2

Re: Large files > 2GB

Any chance the splice functionality will be included into the html5 uploader?  This would give html5 unlimited filesize upload capabilities as well.  Looks like Firefox 4+, Chrome 6+, & Safari 6+ will all support.  MooTools has already incorporated this functionality.  Good reading: http://www.mooforum.net/help-f26/send-f … t4554.html

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

splice or slice? slice method was around for quite some time and Plupload has been using it from the beginning. How is that thing you suggested different from what we do?

Not sure how it may let use to upload file of any size. It still requires the whole file be read in memory, doesn't it?

If you want to see your issue fixed, do not report it here, do it on - GitHub.
  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

Uploading huge files in HTML5 is now possible in Chrome. Gecko has some problems with it, that are now fixed in Gecko 7 and are expected to go wild in September. Next version of Safari may also get it. IE and Opera lack different bits just yet.

If you want to see your issue fixed, do not report it here, do it on - GitHub.

73 Reply by karen66 (edited by karen66 2011-10-25 07:26:07)

  • Registered: 2011-09-02
  • Posts: 1

Re: Large files > 2GB

Very informative thread, I knew much new information on the topic, thank you!!!

  • Registered: 2010-06-14
  • Posts: 20

Re: Large files > 2GB

davit wrote:

Uploading huge files in HTML5 is now possible in Chrome. Gecko has some problems with it, that are now fixed in Gecko 7 and are expected to go wild in September. Next version of Safari may also get it. IE and Opera lack different bits just yet.

I've upgraded my application to the latest Plupload version and latest Chrome version.

Chrome crash "aie aie aie" error if I upload a +400Mb file.

  • davit
  • Maintainer
  • Offline
  • Registered: 2010-11-29
  • Posts: 2,143

Re: Large files > 2GB

@NextOne this feature is not included in Plupload 1.4.3.2 (the latest one on this website). HTML5 support for huge files is currently available only directly from github. Are you using a version from there?

If you want to see your issue fixed, do not report it here, do it on - GitHub.