Living in text files

by Giles Turnbull

A while ago, I thought I'd try an experiment: could I organise all my work, all my personal stuff, all my writing, in one huge text file?



I tried it. It wasn't easy. While it's sort-of comforting to know that you have everything you need at your disposal, it's also a little daunting. "Where did I leave that half-finished snippet of writing about such-and-such? I hope I can find it somewhere in this 4,000-line file..."



Of course, such worries are baseless because any half-decent text editor can find exactly what you need in a file that big, or 10 times that big. You just need to remember what it is you're looking for.



My experiment ended with me abandoning the idea; I decided it was easier (especially with the advent of Tiger and Spotlight) to use separate text files.



And then, a few weeks ago, I purchased a second computer, and had to change my thinking yet again.



With two computers to work on, the question of managing a bunch of essential files becomes more complex. Which machine should they live on? How should they be kept in sync with each other? I played about with a variety of sync methods and couldn't find anything that I felt really comfortable with.



So I switched back to one-file-fits-all; now I only have one text file to worry about, to keep synchronized, and that's much easier.



One thing I learned from the first attempt at using this system: don't try to scroll around from place to place in the file. You'll end up scrolling for hours and losing track of what it was you were scrolling to. It's much, much better to use the find controls. I've started getting in the habit of "tagging" parts of my file with keywords that might come in handy when I'm searching. Using a split view in my editor means I can move items around from one chunk of the file to another, just as if I were copying and pasting between two separate ones.



As Danny O'Brien discovered during his research into effective organizational habits of geeks, text is the simplest, most platform-independent, fastest-to-search format we have for storing information. So everything I need - from todos, blog posts in progress, article ideas, addresses, my list of books to read, the shopping list, and much more besides, lives in just the one file. In effect, I live in that file. When I'm sitting in front of my computer, it feels like home.




Have you tried the one file approach?


30 Comments

jcn
2005-08-09 18:11:56
MacJournal
Why not just use MacJournal (or something like it)? One [proprietary] file; all the text you can ever write. MacJournal is now Spotlight compatible. It will also backup your data to an XML file and/or as individual RTF files. I couldn't live without it.
macnetjournal
2005-08-09 22:41:36
MacJournal
You answered your own question - using a proprietary solution freezes you into that program. Text files work on Macs, Unix, Windows, Palm, you name it. A proprietary solution works on one platform. That isn't flexible...
macnetjournal
2005-08-09 22:50:03
What text editor are you using?
Giles, what program are you using as your text editor on your Macs? And are you using an automated tool to sync your files?


Great topic!


Rob

gilest
2005-08-09 23:45:10
What text editor are you using?
BBedit.


I tried a bunch of different sync tools but didn't like any of them. I want to be able to edit my file on either or both of my machines, many times a day.


I thought of the option of keeping the file on a remote server, which removes the sync problem. But it makes writing too slow; there's a lag between me hitting keys, and letters appearing on screen.


Besides, I often work where there's no network, so I need to keep things local.

Andreas_Bachofen
2005-08-09 23:47:53
Using LaTeX-Tags
Hi,
if you use SubEthaEdit, you can use the LaTeX-Tags for sections or subsections. Just write \section{Some descriprive title} and SubEthaEdit recognizes this as a section-title so you can browse easily via the menu just above the text.


Perhaps this works with BBEdit too, I just don't know.

gilest
2005-08-10 00:05:50
Using LaTeX-Tags
The Markers feature in BBedit provides broadly similar functionality, but I like the sound of the way SubEthaEdit does it.
jmenard
2005-08-10 06:35:52
EmacsWiki
I'm an Emacs-head. For a few years now, I've been using an EmacsWiki (http://www.emacswiki.org/cgi-bin/wiki/EmacsWikiMode), which is an in-editor Wiki. It's the best solution I've ever used: it's text, it has hyperlinks within the editor, I can launch URLs, and I can use all the power of Emacs. I use rsync to copy the entire Wiki from my home system to a server and back. I keep sensitive information in an encrypted Wiki page.
bici
2005-08-15 08:08:52
spilt views
You say 'Using a split view in my editor means..." Are you doing this with BBedit? If i so i can't seem to get this efffect with BBedit (8.x)... how do you get this get his feature/ I would love to also this.
PS I have maintaine on big text file foa few years now, but mainly for all my "bits" like pswds, urls, and other snipptes of info. I keep it on my server and so is available from any where/machine. But i lioke your idea of adding even more to it and then syncing amoung machines. great article.
ASCII: because it still doesn't suck.  
;-)
gilest
2005-08-15 10:27:58
spilt views
Ah yes, BBEdit's split view control is less than obvious. I used the app for a year or more before I even realized it was there.


Open a file and look at the scroll bar on the right side. Right at the top of the scroll bar, and just underneath the toolbar (or what BBEdit calls the Status Bar) is a tiny grey rectangle. Drag it downwards. Ta-da! Split view!

dakegra
2005-08-17 13:24:43
windows text editor
anyone recommend a good windows text editor?


I'm quite fond of TextPad, but am open to suggestions...


Bellator
2005-08-17 13:49:14
windows text editor
UltraEdit is my favourite when it comes to writing long stuff and coding.
For notetaking I use FlashNote, as the hot key is very usefull (Saves me from having to hunt the right window etc)
jmason
2005-08-17 16:13:47
vim and folding
I've also been doing this -- in particular adding del.icio.us-style "tags" beside stuff I'll want to be able to find in a month's time -- like the 'cdrecord' command to burn a DVD+RW backup on linux, or so on.


BTW, vim has some useful support for "folding" -- essentially treating indentation like an outliner would.


As long as you can turn on and off folding, this means you can hide entire blocks of text which share the same indent level. It's still there -- just hidden, and you can still find it in a search. nifty.

mcg
2005-08-17 18:15:16
windows text editor
VIM and Emacs both run on windows
jchoyt
2005-08-17 19:36:14
windows text editor
The most powerful text editor I've found is jEdit (http://jedit.org). The search is unrivaled in any other editor I've tried and would be hugely useful for this approach - especially the HyperSearch, where you get a list of ALL the matches and can quickly click to navigate around a file. Can handle truly HUGE files as well. Java based, so works on other platforms, too.
dice1976
2005-08-18 11:39:19
Text files
On your idea of using one large text file- I find it to be so useful. In fact I'm using the same text file for both work and personal. How do I do this you ask? I use EverNote [http://www.evernote.com/en/]. It's a great text document database that stores all your notes with categories.


I also tried using WikiPad [http://www.jhorman.org/wikidPad/] which is great if you know how to tyep WikiWords so you can link all your documents together.


Both provide the best in text file documentation and notes. I'm still split 50-50 as to which one I am going to use permantely, but EverNote seems to be taking the lead for now.

jcn
2005-08-18 19:05:09
MacJournal
Well, but I also said that it automatically backs up your documents to an (as in one) XML file and individual RTF files; either of which will work on most platforms.
ranh
2005-08-18 23:58:28
Archie is just what you need for single-text-file working
However, I'm pretty sure this doesn't work on the Mac right now, so this may only be relevant for Windows users.


Archie is what became of Jeff Raskin's THI (The Humane Interface), and its essentially a completely re-thought-out text-editor that uses what can be called a single text file (to be fair, it does more than edit text), with "page characters" and "document characters" to indicate where one document ends and the other begins in its one big flow of text.


It has an extremely powerful incremental search mechanism as a replacement for every kind of navigation - you never need to use the mouse, and you select text in just the same way you navigate. Everything you delete is stored in special document, and you can scroll through and bring back everything you ever deleted - and that there also takes care of cut and paste. Not only text, but even previous selections are remembered, and you can go back to them or combine them.


You Everything is done with the keyboard and is command based; scripting and configuration is done in custom documents.


Anyway I'm going on... It's really quite something and I recommend that anyone with a free hour on their hands take a look at the demo screencasts and try the thing out - it changes a lot of what you take for granted with GUI text editors, and would definitely make the troubles of a single-text-file-workflow irrelevant:


This is the homepage: http://rchi.raskincenter.org/aboutrchi/index.php


And here are the screencasts: http://rchi.raskincenter.org/aboutarchy/demos.html

ranh
2005-08-19 00:17:04
ArchY is just what you need for single-text-file working
ofcourse I meant archy and not archie.
joebeirne
2005-08-19 14:24:50
Text Only
I am extremely absent-minded and have done this for many years, first in little pocket Daytimers notebooks and then on a succession of Powerbooks (180, 2700, Wallstreet, Pismo, G4...). The point is to be scrupulously linear in the system: one page has to follow the others. I believe that this is related to David Gelertner's proposal for a stream-of-consciousness file system that would categorize everything not within any spatial or hierarchical system, but exclusively by temporal markers. His argument if I recall (ha!) was that this is how the brain works and so should a computer work - to fit with our brains.


By the way, I still use the little Daytimers notebooks. I go through about six a year (I have small handwriting) and they are the closest thing I have to a diary. At the end of every week I transcribe the new pages into my big text file.


(The only problem being: some weeks never end.)


I use whatever the default text editor is on the platform I am working on. That is critical too for me as I often work far away from my own stuff. I store it all on my 1G Shuttle Ipod, which has no music on it... and may never have any.


I do also use a contact manager (address book on my PB) and a calendar, but I have been fooling around with Backpack to replace iCal for that purpose.


I am not suggesting that anyone necessarily should emulate this system: this is more of a memory replacement than memory enhancement device for me, but if it works for me it may work for normally-brained people as well.


cornell
2005-08-21 04:54:07
yes! one file, emacs, and some WikiWeblogPIM ideas
Thanks for the great article and discussion. I'm a 'one-big-file' user (edits via Emacs), but I've structured the file a bit to adopt ideas from PIMs and Wikis. If you're interested, I just posted about it at http://ideamatt.blogspot.com/2005/08/my-big-arse-text-file-poor-mans.html


matt

richardnewman
2005-08-22 21:11:22
Version control
Something like darcs can follow changes in two different repositories at once, and merge them safely. Real conflicts (editing the same line in both files, say) can be resolved by you.


Darcs is awesome for distributed working -- you can push or pull changes from either location.

MarcS
2005-08-25 15:54:18
windows text editor
I use Crimson Editor.


http://www.crimsoneditor.com/

Chris_Cooper
2005-09-24 06:58:19
Writing in a single text file
[The following is an almost unedited copy of the note I've just made to myself in a new file (on my PDA, a Psion 5mx, which may explain concerns with file size, etc). The point of it is that it's not concerned with getting Life, the Universe and Everything into a text file - just the one small corner of it that consists of what I write and what I don't write but ought to, want to, need to. - Chris]



Journal 2005
Sep


24th


Sudden brilliant insight: not living out of a text file, but writing into a text file writing everything into a single huge Word file. Drafts of blogs, emails, notes, random thoughts complete copies of things of interest, emails from others, stuff from mailing-lists everything I've always thought belonged in a commonplace book.


Questions:
how big can the file get?
what formatting should I allow myself?
what doesn't belong?
graph daily input?
can this fill the functions of a journal?
will I do even less reflective writing?
what are the foreseen pros and cons?


Tell 43folders! (Or is it a forum at a different site?)


It's incremental! It's bottom-up! ...



[Hello to everyone, by the way - this is the first time I've contributed. Sorry if this isn't techie enough for this forum.]

gilest
2005-09-24 15:08:43
Writing in a single text file
Nice idea, Chris. I've also given thought in the past to having a file for recording everything as it comes into my head. I never actually tried it because I feared I'd spend all my time typing instead of actually *doing*...
wow_gold
2005-12-12 00:51:36
MacJournal
world of warcraft gold (http://www.sitgame.com)
wow_gold
2005-12-29 20:25:43
world of warcraft gold
world of warcraft gold (http://www.sitgame.com)
Tom
2006-03-28 20:46:23
been doing so for years. I'm up to about 2MB now, and still find what
I need - I also tag sections. I live in vim, and use the folding feature. Also setup a cron
job to chck the file in to RCS once per day.


I have embedded "files" inside the huge text file - I've sort of reinvented
a crude file ssystem :->. These files w/in the file are delimted with a special start and stop tag. Within these special sections I have multi-line blocks that have a "tag words" delimited by colons as their 1st line. THese blocks are delimited by {<[ and ]>} A perl
script then extracts these "file with in the huge file" sections and flattens
the blocks into a single long line of text, so that I can grep on the tags -
the bash/grep script then reconstructs the \r's in the section so it expands back
to multiple lines.


Rob McCormack
2006-04-09 21:05:49
What an interesting article. I often wonder about how one large file might work.


The one large file idea is behind the reason I created
http://toolbartogo.com - still in BETA, releasing May 2006


It has at its heart, on file called My Notes.


My Notes can be updated via any browser, and computer, any where.
See demo at:
http://toolbartogo.com/indexl.php
Saving is disabled.


It is free to sign up for your own account.


Comments/suggestions are welcome.


Rob

kllk123
2006-05-01 16:51:04
Would you like to have a stronger Character in WOW?
Hello, it will take 16 days from 1-60, and the price will be $177 including some blue equips, even a few purple equips. The gold got in the levering will all belong to you and not less than 200G. It will include 300 first aids and a mount. I'm sure you will be satisfied with it.

click here http://www.gamesgoto.com
Andy
2006-06-01 01:41:49
Try TiddlyWiki - all in one file and it solves your scrolling problem.


http://www.tiddlywiki.com/