Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> huge 3 GB file (3M 1K lines)

I looked into text editors for handing very large files several years ago. For your perusal my results, hopefully not too outdated: [0]

.

SOME USEFUL SPECS to consider

* Performs with %yourmemorycapacity% and %yourdiskthroughput%

* Optionally disable features that multiply required memory capacity and disk capacity & throughput: Auto-backup, temp files, auto-conversion to different character encodings (e.g., 8-bit to 16-bit)

* Optionally disable parsing of data (for any purpose, from syntax highlighting to structured data)

* Load subset of file into memory at any one time

* Begin editing before file fully loaded

.

SOME USEFUL RESOURCES

* http://texteditors.org/cgi-bin/wiki.pl?CategoryLargeFileHand...

* http://www.knudvaneeden.com/links/file/addactivity/computer/...

.

SOME USEFUL SOLUTIONS

* EmEditor: This was very impressive. The speed is breathtaking for what it does, and it can handle advanced functions such as column operations on the full file with ease. We chose this and users were very happy; it transformed their abilty to work with large text files.

----

These were on our shortlist to test next but we didn't get to them due to available time and EmEditor's fantastic results:

* VEDIT: Designed for large files. They charge for it, IIRC, but based on its reputation it is worth the investment.

* The Semware Editor (TSE): Also a great reputation

* 010 Editor: http://www.sweetscape.com/010editor/

* PilotEdit: http://www.pilotedit.com/

----

Also of interest:

* File Query: Treats file as a database, enabling parsing, queries; very flexible and powerful, per reviews: http://www.agiledatasoftware.com/

* PDT-Windows: A database editor which reputedly has a max filesize of 18 EB (exabytes)

* bvi: Binary VI

[0] Some of this is copied from an earlier HN comment of mine



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: