Offtopic, but I use org mode for my notes files (both general and technical), and I've never really figured out how to use it effectively.
My pre-org notes file was just a giant text file with entries delimited by manually-entered '--------------' lines. I stuck a topic in each entry for searchability and then the rest was freeform text. It worked pretty well.
org mode gives me more power, but it's getting a little unmanageable. I find myself expanding and collapsing sections a lot, especially when just trying to find the right place to add a new entry. (I'm using a hierarchy rather than the flat append-only list.) The file is pretty huge at this point, and I split it up once but that broke my most common usage of just isearching through the entire file, and it was annoying to have to load multiple files to do a complete search. I had links between files, but that didn't help for searching.
I imagine there are tips and tricks to resolve all of my issues and nuisances, but I've never taken the time to figure them out. Is there a good workflow description somewhere that I should be looking at? It's hard to beat a basic text file, but after a decade or so it has gotten pretty big and I'd like to eg be able to cluster related things together so I can purge obsolete stuff.
Maybe my problem is that org mode is intended for organizing things, and my use case is too free-form?
Scaling your system is always going to be a problem, no matter what tools you use.
I hear you on isearching. To support multiple files you could look at ag.el for fast cross-file search (although not incremental like isearch).
Alternately you can use :tags: to add a different dimension of information by which things can be grouped. org-agenda can then be used to see items by tag (also since they are just text you can search them in a buffer). I use org-agenda in 'F'ollow mode whereby a second buffer shows the selected agenda item in its original context. It helps visualize where things are while navigating.
The new hotness, ala org-roam, is bidirectional linking with tiny files. That would probably be a pain to migrate to, and isn't really aligned with your free-form model. But it might be the way to scale over the long term.
That sounds promising. You understood perfectly where I am and what I'm looking for, thanks.
I'll check out ag.el. The incremental search is not that big a deal, I'm usually searching for a keyword anyway.
I looked briefly at org-roam, and I agree with your assessment. It looks like it adds friction to the part that needs to be as smooth as possible for me: adding notes. I could probably dummy something up for searching by making a command that shells out to grep or whatever, but that feels like fighting against the data model.
I have a daily.org file I use to capture meeting notes and quick notes into. org-mode automatically keeps it organized as a datetree. I set all my other org files to archive into a subheader of the day it's archived (or done, for todos). So far I like it, but I've only been doing it for a few months. We'll see when it gets to years.
One nice thing is it's easy to see at a glance roughly what I did in a given week or month. It also doesn't insert much process into my flow, which I appreciate.
I had a separate 'daily.txt' file for this sort of thing, and it's desperately in need of a replacement. Your setup sounds pretty good to me. It seems like org mode may be a better match for the daily stuff than it is for my notes file. I especially like the idea of archiving into a subheader of the right day.
Every morning I run `org-diary-new-entry` which inserts a new block. At the end of the day I commit this to a git repository and push it away for safety. I've got a work-log/journal going back a couple of years at this point.
I typically just use capture and refile, almost never enter new things directly. I have a few templates for often-used things (work, personal, funny things my kids say), and then I either just file it directly or if I know it should go under some tree I C-c C-w to immediately refile it there. I use selectrum for completion with selectrum-prescient so my most used refiling goals are at the top of the list, and if I have to filter the list it's typically just two or three characters until I find the heading I'm after.
My pre-org notes file was just a giant text file with entries delimited by manually-entered '--------------' lines. I stuck a topic in each entry for searchability and then the rest was freeform text. It worked pretty well.
org mode gives me more power, but it's getting a little unmanageable. I find myself expanding and collapsing sections a lot, especially when just trying to find the right place to add a new entry. (I'm using a hierarchy rather than the flat append-only list.) The file is pretty huge at this point, and I split it up once but that broke my most common usage of just isearching through the entire file, and it was annoying to have to load multiple files to do a complete search. I had links between files, but that didn't help for searching.
I imagine there are tips and tricks to resolve all of my issues and nuisances, but I've never taken the time to figure them out. Is there a good workflow description somewhere that I should be looking at? It's hard to beat a basic text file, but after a decade or so it has gotten pretty big and I'd like to eg be able to cluster related things together so I can purge obsolete stuff.
Maybe my problem is that org mode is intended for organizing things, and my use case is too free-form?