Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: SHA1 Checksum Large Files with HTML5 Typed Arrays and Web Workers (antimatter15.github.com)
47 points by antimatter15 on Oct 29, 2011 | hide | past | favorite | 7 comments


Cool concept and a great usage of WebWorkers. I can see this being useful to verify the integrity of files uploaded via an HTML form - get the SHA1 client-side and verify it against a SHA1 server-side.

Unfortunately this implementation is way too slow for large files. In Chrome 15 OSX running a Core 2 Duo 2.4GHz an ~650MB file takes about 9.3 minutes to calculate the resultant SHA. Using openssl the hash generation takes ~8 seconds. (Bearing in mind that it's not fair to compare an interpreted script against a compiled program.)


I think this is a fair comparison and the truth (that many cannot swallow) is that Javascript is simply too slow for non-trivial computations. All these talks about Javascript finally catching up in speed to native, and now a slightly non-trivial algorithm and Javascript falls flat in its face. I got similar result to pygorex in my run, and the Javascript implementation is indeed 60X slower than native.

Now compare that to HashSum, a a Silverlight-based implementation of the SHA-1 (I verified that it generates the same result), I clocked it and it is 20X faster than the Javascript implementation, and ~3X slower than native. Try it at http://www.hashsum.com (I wrote it; Silverlight required; sorry about the ad/layout, it was an "exercise" a while back).

So HTML5/Javascript is the future, and the future does not include Flash/Silverlight. Perhaps. But this experiment shows that Javascript is clearly way too slow compared to native or plugin. You could argue that the plugin-based scenario is going out of favor. Politics, competition, distribution, etc, are all valid reasons. But "Javascript is now just as fast" isn't one, because it is not. Those who are gung-ho about how fast Javascript is needs to write their own SHA-1 implementation that beats http://www.hashsum.com ... or at least not be blown away by it.


Browsers have not yet fully optimized the performance of typed arrays (a month or two ago they could be slower than normal arrays):

1. http://blog.n01se.net/?p=248

2. http://code.google.com/p/chromium/issues/detail?id=70675


in IE they have a CAPI activex which allows javascript to call Win32 Crypto API.

I think it would be cool if HTML5 could include something similar. You know, native speed md5, sha1, secure random number, HMAC , etc.


In worker.js :

//Firefox doesn't support FileReader inside WebWorkers

I noticed that Firefox has FileReaderSync available only from Web Workers. https://developer.mozilla.org/en/DOM/FileReaderSync

Per spec (http://dev.w3.org/2006/webapi/FileAPI/#readingOnThreads) web workers can use both. The use cases I have will work fine with the sync interface, but I wonder why Firefox chose not to provide the async option in workers.


OK, this is pretty cool :-)

If you don't mind my asking, what's the license on this, if I'd like to use this in one of my projects?


Wow, this is awesome. I prefer using a desktop program for multiple checking, but this is very nice looking. Thanks for the resource.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: