It has not escaped my attention that there has been some comments on various fora about my review of Scrivener as a Note Taking program, so perhaps some clarification is in order.
I don’t normally group all my notes together, normally they are all grouped by subject in separate files, that is why interconnections between files are so important.
The test with lots of files was just that, a test, load the program until it breaks and see what happens. If the program doesn’t break then you can be confident in using it for any normal sized set of data, and if the program does break then it usually reveals interesting things about the program.
The comments I made in the review about Scrivener slowing down when working with ten thousand notes was not really a criticism merely an observation. It is a good thing that Scrivener works at all under these circumstances.
These are not trivial documents, they were downloaded from Project Guttenberg, the smallest is only a few kilobytes but the largest is two and a half megabytes. The whole collection is around three quarters of a gigabyte. I used these documents for the test because they were convenient, I had them for another purpose but whilst I was reviewing the note taking programs it was convenient to just import some of them to see how the various programs performed under stress. The point is to stress the program until it breaks and see what breaks. First with a hundred documents, then with one thousand, then two thousand, then five thousand and if the program is still working throw the full ten thousand at it.
Scrivener performed very well with three quarters of a gigabyte of notes. The automatic backups were slow but that was the only effect, this is unsurprising given the amount of data and the fact that saving data to disk takes time. Searches slowed down but not by as much as one might expect.
With MyInfo the only effect was that that the notes file took a long time to load and save, again unsurprising given the amount of data to be saved and loaded. But MyInfo doesn’t do automatic backups so it may be that the data is less secure. Searches were still lightning fast and the overall performance did not slow down noticeably.
At the time I reviewed ConnectedText I only tested it with about a thousand notes and experienced no problems, this was my first review. ConnectedText is one of the programs I continued to use so later I loaded the full set onto it just to see how it would cope. With the full set of documents ConnectedText became slow in some aspects of its performance. Searches in ConnectedText slowed down to the point where using the program was difficult, also it sometimes ran out of memory whilst doing a search, I think the search engine (and probably the rest of the program) is only 32 bit. Also I put an index of documents grouped by author on one of the pages, the rendering of this page slowed down to the point where it was very inconvenient to view. But to be fair ConnectedText has some very sophisticated facilities and this was using a facility not present in the other note taking programs.
Memo Master performed quite well with about two thousand documents, I did not test it with ten thousand documents.
WhizFolders was another of the programs I continued to use after the review, until I looked at MyInfo which has taken over from it for everyday use. WhizFolders performed well with one thousand documents but with ten thousand the load and save times were quite long, searches became slow and there was a few seconds delay when adding a new document.
Debrief Notes slowed down considerably with just one thousand documents. It is not a good program to use for other reasons and no further testing was done but it would probably have slowed down still further if more documents had been added.
LexiCan slowed down to an unacceptable degree with only four hundred documents. This seemed to be related to the size of the documents. There would be less slowing with short documents than with large documents.
Essential PIM is my current e-mail program. I tested it with a thousand documents and it didn’t slow down very much. I didn’t test it with any more than that and deleted the documents soon after that. I don’t use this program for note taking but it can generate links so that I can have links to e-mails from MyInfo and ConnectedText.
Personal Knowbase didn’t slow down with a thousand documents but with ten thousand documents the load and save times went up considerably, much more than would be expected. Once it was running the performance was good, even with ten thousand documents.
Keynote-NF slowed down with only two hundred documents. With one thousand documents it became unusable. The slowing seemed to be proportional to the size of the document rather than their number. A small number of documents containing high resolution images virtually crippled it.
Microsoft OneNote was not tested due to the lack of a bulk import facility.
MyBase slowed down quite noticably with one thousand documents but also there seems to be a physical limit on the size of the database, if you exceed this limit the program crashes. The limit is approximately 300 megabytes but it may be lower than this.
I hope this clears up the point that I was not advocating that people should put all their notes into one file.