Friday, September 30, 2011

Hölderlin and Version Control

The German poet Johann Christian Friedrich Hölderlin (1770-1843) wrote his first drafts usually on the right half of the right page of a folio sized notebook (Heft). He left the back of this page empty, so he could use the left part of the page and the back side of the previous page for revisions and reformulations. He thus always had three times the space for later work than he used for the first draft.

Sometimes he wrote a rough sketch on the lower part of next page that was separated by a continuous line from the rest of the manuscript. As this sketch was worked out, he transferred its contents to the upper part of the page.

This approach was not unusual for the eighteenth century and various versions of it survived into the twentieth century. Thus, I was taught (late fifties, early sixties) to divide every page in a notebook in half by folding it (and then flatten it again). The first draft was to be written on one the right half of the page. The other half was to be used for revisions. What makes Hölderlin's approach remarkable is that he planned for three times the revisions (and apparently used up the space as well). Among other things, this shows that his poetry was not the result of sudden inspiration, but of gradual reworking of the first draft. The editors of the Frankfurt Edition of Hölderlin's work like to speak of "ideal growth."

Whether "ideal" or not, this kind of growth is difficult to see in an electronic text, unless, of course, the author used version control. This may not be a bad thing entirely.

Thursday, September 29, 2011

Interpreting Imaginary Slips of Paper

H. A. Wolfson made the following claim about Spinoza's work: "if we could cut up all the philosophic literature available to him into slips of paper, toss them up into the air, and let them fall back to the ground, then out of these scattered slips of paper we could reconstruct his Ethics." He did proceed to reconstruct Spinoza's Ethics in this way, but found in the process it was more like a jigsaw puzzle with too many pieces, with pieces that do not fit together and had to be "reshaped," and with "many necessary pieces ... missing" and having to supplied by "ourselves." But he claims to have a guide or outline of the picture in the form of the Ethics "as it was originally formed in the mind of Spinoza."[1]

It appears to me that this is a good description of how not to interpret an author—at the very least how not to interpret an author of Spinoza's stature. The first problem is that Wolfson (or anyone else, for that matter) has no access to the picture of the book "as it was originally formed in the mind" of any author. Secondly, the idea that you could reconstruct a work like Spinoza's from the source he used or consulted, diminishes it considerably from the outset. Missing and reshaped "imaginary pieces" make the interpretation even more arbitrary.

I am all for the "reduction in order to build complexity" (Luhmann) and I do use imaginary slips of papers (ConnectedText topics) in my research, but this stuff is just that: "stuff." The arguments and story lines based on some of this stuff do not result from the mere addition, subtraction and shaping of preconfigured pieces.

This observation is infinitely more appropriate in Spinoza's case. Wolfson's approach has a definite tendency to reduce him to the lowest common denominator, that is, I am tempted to say, Wolfson's own imagination.[2]



[1] H. A. Wolfson, "Behind the Geometrical Method," Spinoza: A Collection of Critical Essays. Ed. Marjorie Grene. Garden City/New York: Doubleday Anchor, 1973), p. 3.

2. Wolfson has a further means to reduce Spinoza to his imagination. He asserts: "Statements are not significant for what they actually affirm but for the denials which they imply" (p. 17). The complement of a term (or, perhaps better, of "what a statement affirms") is infinite. So there is much to choose from!

Wednesday, September 28, 2011

Important Problems

Richard Feynman seems to have given younger scientists the advice that they should keep a list of a dozen or so of their favorite problems. They should have this list constantly present in their mind. In this way they could relate everything they read or heard to one of the problems on the list and then determine whether the new information could help them in solving the problem.

The claim was: "If you do not work on an important problem, it's unlikely you'll do important work."

I am not a scientist and my problems are not as clearly defined as those of physicists, but I have some definite areas of interest on which I take notes. Every time I notice something that belongs to one of those areas, I take note, enter it into my hyper-textual database and connect it with the stuff that is already in my ConnectedText Projects.

The Art of Choosing Difficulties

Yesterday, I bought Winifred Galllagher, Rapt: Attention and the Focused Life. (New York: Penguin Books, Galagher, 2009) to read on the "T." It's an interesting and easy read. In reading it, I came across a reference to Nicholas Hobbs, whom I had never heard about before. Here a quote from Hobbs (not in the book):
... the healthy person, the healthy in body and spirit, is a person faced with many difficulties. He has a lot of problems, many of which he has deliberately chosen with the sure knowledge that in working toward their solution, he will become more the person he would like to be.

Part of the art of choosing difficulties is to select those that are indeed just manageable. If the difficulties chosen are too easy life is boring; if they're too hard, life is self defeating.

The trick is to move oneself in the direction of what he would like to become at a level of difficulty close to the edge of his competence. When one achieves this fine tuning of his life, he will know zest and joy and deep fulfillment.

This seems right to me—not just for life in general, but also for research and thus note-taking.

Wednesday, September 21, 2011

ConnectedText 5 Released

Today, ConnectedText version 5 was released. I blogged earlier about what I especially like in the new version. There were two things that I did not mention because they were not part of ConnectedText then. The first is automatic category assignments (or automatic tagging), the second is the addition of sorting in attribute and property summary. This was a direct response to this post.

You can now add a "+" or "-" sign to a column in summary search and force an ascending or descending sort of the information on the basis of that column.

[[$SUMMARY:Task|priority|what|-when|done]]

will result in this:

Click to enlarge!

See the little down arrow after "when" It can easily be changed to up or a different column can be used for sorting, like so:

[[$SUMMARY:Task|priority|what|when|+done]].

Miles said... about the previous post: "Very good. It's a shame - especially for a Schedule Topic like this - that CT doesn't let you choose a Sort order."

I asked whether it could not be added. Eduardo fulfilled the wish. Now that's what I call a quick response. I have never seen a developer more responsive to his customers.



P.S.: Perhaps I should add that I changed the summary table in the CSS file to get the appearance I like. It's now this:

/*
** Summary table
*/

table.summary {
margin: 1em 1em 1em 0;
background: ;
border: 0;
border-collapse: collapse;
}

.summary th,
.summary td {
border: 0;
padding: 0.2em;
}

.summary th {
background: ;
text-align: left;
}

.summary caption {
margin-left: inherit;
margin-right: inherit;
font-weight: bold;
}

.editsection {
float: right;
margin-left: 5px;
}


Postscript: ConnectedText 5.02 shows the dates in summary view like this "09/28/2011". This makes the dates more readable—and more like Agenda's dates as well. (Wednesday, September 28, 2011)

Sunday, September 18, 2011

orGenta

There is another Agenda wannabe. It's called orGenta. It has the look and feel of a real spreadsheet. Since "orGenta stores your Items and Notes in a Microsoft Access database, ... you may run queries in Access, or link to your data in Excel."

orGenta has automatic category assignment and "smart date recognition." I tried it and found it less than intuitive. This may have had to do with the fact that I did not give it enough time and the "feature" that "items get assigned" to categories "in the 'background': you may not see them applied until you expand or contract the TreeView." I found that disconcerting.

I mention it here for completeness' sake, as I have talked about other Agenda-like applications before. In many ways, this is the most "faithful" to Agenda as a "spreadsheet for words." If only it would look less like a spreadsheet.

It's shareware and costs $10.00.

There has been no development since 2006, but it might be worth a look for hard-core Agenda aficionados. I do not belong into that category (no pun intended).

Saturday, September 17, 2011

ResophNotes 1.2.1

ResophNotes is now available in version 1.2.1. It has the following changes:
  1. Support pinned note feature
  2. Support single note mode (F11)
  3. Recognize email address as link
  4. Fix problem for link for selected text
  5. Fix encoding "," in text file name
  6. Allow simple MarkDown export (optional).
I like the second new feature the most, as I only need it for temporary notes that end up later in my main application.[1] See this:



1. See also this, this and that.

Agenda and ConnectedText, V

Here is another way of doing scheduling that is sort of like Agenda's way of doing it. It involves Python scripts and Attributes. This way has the advantage of collecting items from within many different topics. So you could put items like this "[[task:=20110924 - Revise Chapter VII]]" into any topic and have them appear on your Scheduling pages.[1]

This relies on a slightly tweaked script written by Rob Hughes (available here. It was meant for a GTD application but can be used for many other purposes.

On the scheduling page, you insert:

**[[Tasks]]**
<% Python from gtd import * print_actions_for_contexts(("task",)) %>
**[[Ideas]]**
<% Python from gtd import * print_actions_for_contexts(("idea",)) %>
**[[Plans]]**
<% Python from gtd import * print_actions_for_contexts(("plan",)) %>

The result will be something like this:


Here you have just two columns, namely the name of the attribute and the link to the topic in which the attribute is found. You can get creative by naming your attributes to reflect what would be in other columns, if you used the previous solution. You can also enter your your items into topics that are named to reflect their priority or their project. And finally, you could just put the additional information with the items (attributes) into the topic that contains them. that is the beauty of links that they take you with one link to the referring page.

In any case, I hope to have shown that there are more ways than just one to reproduce Agenda-like behavior in ConnectedText. If I were a programmer and not just a hacker, I could probably vastly improve on this capability.



1. Needless to say that I use AutohotKey to abbreviate stuff like this.

Tobu

Tobu is an information manager written in Python that relies on columns, very much like e-mail programs, which in turn seem to be indebted to spreadsheets. It has therefore some similarities with Agenda.

Its main constituent are titles, tags and views. You enter items by name and then add tags. The tags are shown in the view as columns. some tags are special, they result in sortable columns. The tags for sortable columns are written like this "cost: 100" or "chapter: 01", and the value after the colon is used to sort the view. "When Tobu sees a sortable tag, it inserts the value of the tag in the column under that tag; when we're dealing with regular tags, an 'x' is inserted to indicate that the tag is present in the record, an empty space indicates that it's not present. You can select the columns you want to see from the top and the program will show just the items who possess the tags you have typed into the box." Tags are sorted alphabetical or numerical, but there "are a few special values that will be sorted in a different way: short month names, short day names and priority descriptions - 'low', 'medium', 'high', 'urgent'".



Titles are like items in Agenda, tags like categories, views like views. Clicking on a column head of a sortable tag will order the view in accordance with this tag. Views in Tobu are really just filters. This is probably true of Agenda as well. However, the tags constitute a flat list, not a hierarchy as in Agenda. And there is no automatic assignment of categories. Nor completion proposals.

The grammar of sortable tags is not that different from attributes or properties in ConnectedText.

The claim is: As the number of items grows to 5k, 10k, 50k and more, it becomes exponentially harder to find a set of relevant items; it's also harder to enter items in the tree structure because you have to choose the most important category among many possible ones. Full-text search helps but it will often produce a set of results that is too big or too small - it is particularly difficult to use search if you're looking for a set of items instead of just one item, and you won't be able to sort results as you may do with Tobu.

It's an interesting project that has possibilities which for the most part are unrealized yet. At this point, I find its concept more important than its usability.

It's donation ware and it works on Linux as well.

Tuesday, September 13, 2011

Agenda and ConnectedText, IV

In the last post of this series I showed how yuou can easily reconstruct Agenda's capability on different views on your research data and promised I would next deal with scheduling and planning.

There are several ways of reconstructing this affordance in ConnectedText. Here the first one, using attributes. ConnectedText allows you to assign attributes to text in different topics. Attributes look like this

[[Priority:=high]], [[Priority:=medium]], [[Priority:=low]]
[[What:=Task]], [[What:=Idea]], [[What:=Plan]]

and whatever else you want. I am sure you get the idea.

In each topic you want to appear in a topic used for scheduling, you might add:

[[priority:=high]]
[[what:=Task]]
[[when:=20110903]]
[[done:=20110903]]

It is easy to retrieve and summarize this information in another topic. Let's call it "Schedule" and enter this:

**Tasks**
[[$SUMMARY:Task|priority|what|when|done]]

**Ideas**
[[$SUMMARY:Idea|priority|what|when|done]]]]

**Tasks**

The result will be this, provided you have created the appropriate topics or items:
It does not look quite like the original reconstruction, but it is close enough and I believe it is better, since you can add a lot more columns in this way.

Obviously, it is better to create a template for items that are to be used in a schedule rather than adding the details "by hand." Luckily, ConnectedText has templates.

This is not the only way to achieve Agenda-like scheduling. Nor is it perhaps the best way to achieve it. In this method, items are identical to topics. There is a way to slice and dice the information even further and look at parts of topics. We will take a look at that approach in a future post.

Sunday, September 11, 2011

Notetab Outlining

Notetab Standard and Notetab Pro have an Outliner of sorts. It basically consists of a flat list of topics which does not seem to allow different levels of notes.

You can however emulate different outline levels quite easily. Go to View|Options|Outline and select "Show heading in Outliner." You will now see the headings at the top of the topics. Adding a space (or two) at the front of the heading will actually show the text as if it were the child of the previous topic. Two (or four) spaces make it appear like the child of the child, etc.[1]

It's not easy to keep track of the spaces, but it will work.


1. Actually, you do not have to make the headings visible. You can also add the spaces directly when you enter the topic name. But I find it easier to keep track of them with the headings visible in the topic.

Monday, September 5, 2011

Lotus Agenda and ConnectedText, III

James Fallow's observed in his original review of Agenda what he valued most in the program was a function he "didn't know it possessed when [he] first bought it," namely its ability "to organize and keep track of large quantities of research information." This is where ConnectedText really shines.

As Fallows observed:
Suppose you had collected information on your computer in dribs and drabs, knowing that eventually would need to put it together in some organized way. Perhaps you had been copying out notes and citations for a thesis ... Agenda can then sort the information into usable categories, based on the rules you specify.
The same thing will hold for ConnectedText 5 which will have automatic category recognition. The difference is that you do not have to specify rules, but that the program will do so automatically, based on your past assignment of categories. This does not, however, prevent you from assigning categories at will either. (How do I know? I was a beta tester.)

But categories are only one way to organize the information. You can also use smart topics, as I suggested in my previous post see II.

Let's take Fallows' example:
If your project concerns the history of China, you could specify that any paragraph containing the words "John King Fairbank" could be assigned to categories such as "Harvard scholars," "long-term impact of American missionary families," and "Who Lost China controversy." If you were research world leaders, you could assign all names including "King" to the "royalty" category, except those also including the words "John Fairbank," "Martin Luther," or "Kong."
He admits this is time consuming, but finds that "occe you have laid out" these categories, "Agenda can retroactively apply them information already on your computer, and automatically categorize each new bit of data you add."

The pay-off is that you will then be able to define views in Agenda and can as a result
switch from view to view and see exactly the information you are looking for. If I want to see all items concerning relations between America and China, I bring up one view. If I want to see all citations I've collected from the writings of John King Fairbank, including some that were also in the "America and China" view, I can switch there. Then, if I want to see notes on the influence the children of American missionaries in Asia, I can switch to another view and see comments from Fairbank, and Edwin Reischauer, and Henry Luce.
To achieve this in ConnectedText you do not create categories like in Agenda or folders like in Ecco but a topic in which you formulate an inline query. So, if you want a view that contains all items (topics) on America and China, you can formulate a query like: [[$ASK:China & America|INDEX]] (where Index indicates only that you want the material ordered in a bullet list, there are other ways of ordering the results of these searches, but this is not important for the present topic. More important is that ConnectedText does not just allow simple searches, but expressions composed of: [(] [NOT] {AND, OR, NEAR, XOR} [NOT] [)]. It also allows wildcards, etc. Categories, attributes and predicates can also be searched for.[1] Regular expressions cannot be used in inline queries, however.

You do not have to worry about the contents of a smart page. If you add another topic that satisfies its criteria, it will be automatically added to the page. Category pages are, by the way, work very much like smart topics as well. They "search" for pages with categories.

So, as I suggested in the last post, you might want to create a page called "Views" in which you collect all the different pages that you consider as views. You can put a link to that view into the footer so that it is available on every topic page, like so:



Clicking on Views at the bottom might bring up a page like this:


And here is a view of topics that contain references to ConnectedText in my main note-taking file:


This is the easiest part of reconstructing Agenda's capability in ConnectedText. Since this was, however, the most important use many people made of Agenda, it is quite significant already. I will address scheduling and planning in a future post.



[1] More about that later.

Scrolls, Books and Tablets

There is an interesting article in the New Tork Times about what came before the book and what seems to be coming after it.

No further comment!

Sunday, September 4, 2011

Lotus Agenda and ConnectedText, II

Let's look at the "moving parts" of Agenda and ConnectedText and compare them. I will ignore such things as the ability to handle pictures, videos and other non-textual objects. I concentrate on textual matters. I will also leave out automatic date assignments that make Agenda special, as they can be reconstructed with a program like AutoHotkey.
  1. files,
  2. (only one file can be open at a time)
  3. items and notes (it manages mainly items which can have 150 words; you can attach notes by pressing F5; notes can have 1400 words (or about 4-6 pages)
  4. columns and
  5. categories,
  6. views.[1]
ConnectedText has
  1. projects (the equivalent of files; more than one file can be open at a time),
  2. topics (the rough equivalent of items and notes;),
  3. links (completely absent in Agenda); they incluce inter-project links,
  4. categories (that behave a bit different from Agenda's categories),
  5. properties and attribute (which have no equivalent in Agenda) and
  6. search (which allows you to construct "smart topics;" perhaps surprisingly, Agenda had no search function). Search can find topics in any open project


Items: Anything can be an item: appointments, tasks, quotations "or anything else you want to find later on" (Fallows). They are the basic units of information in Agenda. They are small chunks of information, i.e. they must be no longer than 150 words long. This leads to the granular or chunky approach to managing information that I have always liked. Items are similar to the headings of index cards. The notes would be like the rest of the index card, but can hold a lot more information than a 3x5 card could (i.e. 10,000 characters). The same holds for ConnectedText, except that items and notes are more closely integrated. "Item" corresponds to the title of a ConnectedText topic and "Note" to its content.

Ecco also had items, with the only difference that these items were presented in Agenda as a plain list, in which everything seems to assigned the same level of significance. Rather, items are by default presented as an outline. This is the most important difference between Agenda and Ecco. It allowed a more fine-grained characterization of items. Items in Ecco could also be much longer. But this also cluttered the original design and made it more confusing to some who saw in Ecco mainly an outliner whereas it really was a spread sheet with outlining capabilities. (See below.)

Columns: Let's look next at columns. They represent the only obvious difference between Agenda and ConnectedText. And this is no small matter, as columns are absolutely central for Agenda. They may be said to be the "engine" of Agenda. It was was designed to be a "spreadsheet for words." They correspond to the columns in spreadsheets. Items provide the rows. So, Agenda provides you with different cells in a two-dimensional matrix or grid consisting of rows and columns, just like a spreadsheet. Without columns Agenda would be just like any other list manager. With columns it is much more powerful, i.e. it is a spread sheet for Word. Columns can be standard or text, numeric or date columns. (They are standardly indexed, but they can also be made unindexed).

Ecco later copied this approach. It also had columns, but they added check marks, pop-ups and Gantt chart columns.[2] Both allow you to view the information in accordance with different columns. As already mentioned ConnectedText lacks this capability.

Categories: The columns are closely integrated with categories. In fact, they are the main means for assigning categories to items and they are essential for defining views. Views are like different tables in a spreadsheet. Everything in Agenda is a category in some sense. For that reason, they are usually considered to be the central feature of Agenda, but it is important to remember that categories need or are columns as well. It is this that makes them more interesting than ordinary categories or tags. What makes the categories even more interesting is that they are "automatic." In other words, you can specify rules that cause Agenda to apply certain items to categories. The most primitive rule is to assign a topic to a category, if the name of the category appears anywhere in the item text, but you can specify also a short word and a number of other key words that trigger the assignment. You can also specify conditions for the assignments. These assignment conditions have to with the fact that categories can be nested or form an acyclic graph, a tree, in other words. They specify whether an item will inherit the category from a parent or a child in a tree.

Ecco essentially followed Agenda here as well. But categories became Folders with auto-assignment rules. The "auto-assign feature automatically evaluates items throughout your file (PhoneBook, Calendar, and Notepads) and gathers them into specific folders" (Ecco Help File). This shows that categories are a bit like smart folders or folders whose contents are determined by a saved query. Version 5 of ConnectedText will also allow auto-assignment of categories. Categories are, however, a bit different from categories in Agenda simply because they are not connected with columns. Accordingly, they cannot, per se, be used to "pivot" the information spreadsheet-like to open up different views in the way that Agenda can.

Views: This brings us to views. One of the difference between Agenda's spreadsheet-like behavior and a textbase conceived along the lines of a database is that it eliminates queries. You don't have to search explicitly, but you define a "view" in which information is represented in accordance with the categories and columns. This means that Agenda can sort any information you have put into it by categories, which in turn are determined by rules you have specified before. In Ecco views are most closely related to Notepads, which allow you to view the same information from different "perspectives" or in accordance with different folders. There is even one notepad for search results.

While ConnectedText has no folder—everything is an item—it does have the equivalent of smart folders, namely "smart topics." This smart topics are essentially saved searches (as in Ecco), but they are much more powerful and include not just and, or nearby and not, but also RegEx expressions.[3] You can not only search for text in topics, but also for categories (predicates and attributes, about which we have not yet talked) and other things. These smart topics correspond very closely to views in Agenda. In Agenda you have to press F8 to show the different views you have designed and then select one. In ConnectedText you can name one (or more) topic(s) called views, in which you reference the views you have designed. You can furthermore put this topic into the header or footer and have immediate access to it by pressing on it.

Links: The last functionality has, of course to do with the ability easily to link topics (or what would be items in Agenda. all you have to do is enclose the topic in Square brackets, like so [[Views]]. Linking adds a whole new dimension that is completely absent from Agenda (and Ecco). As I have written about Wikis many times before, I will not say anything more than that it can be very useful in reconstructing some of the features directly available in Agenda. (Just see the example above).

Properties and Attributes: You can assign properties to topics, like "priority," "done," etc., etc. Your imagination is the limit. These properties are never displayed in a topic, but they can be used in Searches. You can also define attributes, that is, expressions like "[[$PR:Status:=OK|DateDue:=20080519]]". You can also search for them (and they have their own "smart topics" that display all the topics with these attributes. This semantic dimension was, of course, completely missing in Agenda.

Search: Enough said already. ConnectedText has the most powerful search available in Wiki software. Next time, I will get into how to reconstruct Agenda's capability connected with rows and categories (and then some).

1. There are also sections. they are parts of views and ultimately just categories.

2. I discuss Ecco here as a successor to Agenda, but Zoot is another worthy contender. It recreates some of Agenda's capabilities. Another one was InfoHandler, but it's development has stopped. It can, however, import Agenda files. Still other applications, i.e. those which rely on keywords or hierarchically order keywords, have very limited similarities.

3. It appears that one of the recent add-ons of Ecco, developed by independent developers, adds this capability, but I have not tried it out. In ConnectedText RegEx does not work in inline queries.

Thursday, September 1, 2011

Lotus Agenda and ConnectedText, I

I have been thinking about Lotus Agenda and ConnectedText lately. My question is whether one can re-create Agenda's functionality in ConnectedText. I think it is possible. Here a first step. Compare these two Screenshots:



More later about the why and how.