Querying grouping & sorting across several text files with the FoldingText Command Line Interface

Thanks - I’ll give all that some thought, and sketch out first versions of the queries

I’ve continued to play with this script and am finding it very useful. I’ve run into two more general features that I think would be useful. I regret to say that I am still out of my depth in trying to read/interpret the ViewMenu.json file. I was able to add a report that swapped due dates for start dates, but other than that, I haven’t been able to tinker much and get any results.

At any rate: would it be possible to:

  1. return child lines along with tagged lines? E.g., my files often look like this:

    - task name with tags @tag note for the task (this line should be indented to be a child of above line but forum software not letting me do that)

but in the reports, the note line (the child) doesn’t appear.

  1. display other tags on lines in the reports?

For example, in a report that displays lines by due date, I’d like to also see the other tags that were on the line

e.g.,

original:

- task name @tag1 @tag2 @due(2014-07-22)

would become:

`# Due 7-22-2014

  • task name @tag1 @tag2`

whereas now the additional tags don’t appear.

Thanks!

Hi – things have been a bit busy, have done a bit of maintenance on this, and hope to get back to it on Friday. In the meanwhile:

Q2 (tags)

If $item is mapped to a line, {$item} will just be translated to the ‘text’ part (no syntax prefixes or terminal tags), but
{$item@line} will be translated to the whole raw line, prefixes, tags and all.
If it’s the value of a particular @mytag that you want, you can write:

return: “- {$item} {$item@mytag}”

but it sounds as if I should add something like fn:tags($item) to return the tags of the line without its syntactic prefixes.

Q1 (descendants)

Do you want the full set of descendants of the displayed line, or just its immediate children ?

(If you want to send me the rough JSON of what you want, or the existing template that you want to work from, I should be able to sketch an approach for you )

PS I think that in the version you have, you should be able to write:

return: "- {$item} {$item@tags}"

for:

# Due 7-22-2014
- task name @tag1 @tag2  @due(2014-07-22)

but I’ll need to add an fn:tagsubset of some kind so that we can select which ones to show inline.

I think in practice this would be the same for the way I use it, but for edge cases, I think full descendants would be best.

This is great–you’ve taken my meaning exactly. Thanks for responding to my requests. No worry on the timeline–if you’re busy put if off!

"Started (with ftdoc:// links)": {
		"title": "## Started This WEEK",
		"for": "$item in //@start < {now}",
		"let": "$day = fn:daypart($item@start)",
		"groupby": "$day",
		"orderby": "$day",
		"return": [
			"### {$day}",
			{
				"for": "$i in $item",
				"orderby": "$i@start",
				"return": "- fn:timepart({$i@start}) {$i} {$i@tags} [Link]({$i@url})"
			},
			""
		]
	},

Based on your recent response, this is pretty close to exactly what I’d want (adapted from your “due with ftdoc links” report).

The only other changes that might be nice would be:

  • if “Link” could be replaced with the file name from which the line came (which in my use, is the project name).
  • if tagged lines brought their descendants (Q1 descendants, from above)

Here’s a new query I wrote that returns “@next” tagged items and groups them by their parent item. In other words, if you have lines tagged “@next” in .todo blocks, it returns something like:

## Original project header.todo
   - task name :: [Link](ftdoc://link to original item) @next @read

I use this to search across all my project files for next actions. Would be best if the project header also contained a reference to the filename as well (which, for me, is the name of the overarching project), but I can’t figure out how to get that to display yet.

"Next Actions with Link": {
		"title": "# Next Actions",
		"for": "$item in //@next",
		"let": "$project = $item@parent",
		"orderby": "$project",
		"groupby": "$project",
		"return": [
			"## {$project}.todo",
			{
				"for": "$i in $item",
				"return": "- {$i} :: [Link]({$i@url}) {$i@parent} {$i@tags}"
			},
			""
		]
	},

Looking for a bit of help here. In my “next actions” query above, $i@parent correctly returns the parent line of any line tagged @next. However, why doesn’t $i@child do the same thing? Instead of returning the child line, it returns some gibberish numbers. Here’s the whole query:

"Next Actions with Link": {
		"title": "# Next Actions",
		"for": "$item in //@next",
		"let": "$project = $item@parent", 
		"orderby": "$project",
		"groupby": "$project",
		"return": [
			"## {$project}.todo",
			{
				"for": "$i in $item",
				"return": "- {$i} :: [Link]({$i@url}) {$i@tags} \n {i@child}",
			},
			""
		]
	},

Excellent - I’m hoping to get back to all that on Friday, and I’ll take a look at including filenames then.

Made some improvements to my started query. Now it groups by date, then groups by “project” (the parent header of the returned tasks) and it filters out @done tasks:

"Started (with ftdoc:// links)": {
		"title": "# Starting in the Next 7 Days",
		"for": "$item in //@start <= {today + 7d} except //@done",
		"let": "$day = fn:daypart($item@start)",
		"groupby": "$day",
		"orderby": "$day",
		"return": [
			"## {$day}",
			{
				"for": "$i in $item",
				"let": "$project = $item@parent",
				"groupby": "$project",
				"return": [
					"### {$project}.todo",
					{
						"for": "$j in $i",
						"return": "- {$j} :: [Link]({$j@url}) {$j@tags}",
					},
					""
				]
			},
			""
		]

	},

@parent is a uniquely defined property of a node/line, and you can look at its derivation in the getAttrib() function in the plugin.

@child is not a uniquely defined property of a node (any given node might have various different children), so it’s not one of the properties that getAttrib() can return.

But could you tell me more about what exactly you mean by the the/a child line ? (To Foldingtext its any one of the next generation descendants … but it sounds like you may have something more narrowly defined in mind)

Just trying to bring along the task’s “note”. E.g., I have my .todo sections set up similar to Taskpaper format:

# Header.todo
  - task @next @read
    my note about the task
  - my second task @read

In other words, if a line has a “note” (a descendant? I’m not sure of the nomenclature here), I’d like it to be brought along into the report.

OK, so just the first child line ? or might the note have line breaks, and consist of a set of peer lines ?

yeah, some notes do have line breaks and would consist of peer lines. But in my usage, no “notes” would have children of their own. But, if necessary, I could survive with just the first child line (this covers 90% of my current files and I could easily change practices here to accommodate).

I think we should be able to give you the set of immediate children of type body - off the top of my head, it will probably be by entering one more return loop, and defining that set of children.

(But possibly cleaner to add a @note property to getAttrib(), I’ll take a look – thanks for the work and feedback – it’s very helpful)

Running this quickly through Jesse’s parser, it looks as though notes following a todo bullet are parsed as peers of the list item rather than children.

# Header1.todo
- task @next @read
	my note about the task
- my second task @read


# Header2
- task @next @read
		my note about the task
- my second task @read


┌ Tree
┠┰∅ [0, root]
┇┠ ∅ [1, empty]
┇┠┰# Header1.todo [2, heading]
┇┇┠ - task @next @read [3, unordered]
┇┇┠ ▸my note about the task [4, body]
┇┇┠ - my second task @read [5, unordered]
┇┠ ∅ [6, empty]
┇┠ ∅ [7, empty]
┇┠┰# Header2 [8, heading]
┇┇┠┰- task @next @read [9, unordered]
┇┇┇┠ ▸▸my note about the task [10, body]
┇┇┠ - my second task @read [11, unordered]
┇┇┠ ∅ [12, empty]

I’ll experiment with adding a @note property which harvests immediately following nodes which are body or empty

(editor.tree().toString() is a very helpful function in the FoldingText scripting API incidentally – it lets us look directly at the structure of the underlying tree):

property pstrJS : "
	function(editor, options) {return editor.tree().toString()}
"

on run
	set varResult to missing value
	tell application "FoldingText"
		tell (make new document with properties {text contents:"
# Header1.todo
- task @next @read
	my note about the task
- my second task @read


# Header2.todo
- task @next @read
		my note about the task
- my second task @read
"})
			
			set varResult to (evaluate script pstrJS)
		end tell
	end tell
	return varResult
end run

Cool, thanks! Looking forward to the results.

I’m still just tinkering with cut-and-paste with this (no formal javascript knowledge yet), so digging into Jesse’s tools are a bit out of reach yet. That’s why I appreciate your examples for this plugin so much–they’re rich and easy enough to learn from.

I’ll try to put out a fuller set.

In the meanwhile, Jesse has explained to me that notes are recognised as children of todo items when they have an extra tab indent (I misinterpreted what the tree diagram was showing in the case with an extra tab).

If you see nodePath searching as something which might become more central to your workflow, that means that you might well get some benefits from indenting your notes one tab more deeply under the list items.

(For the purposes of a @note attribute for the getAttrib() function for reports, we can make it recognise either layout, but might be worth bearing in mind that bigger queries (multiple files) are likely to run a little faster if note-children are made immediately recognisable by a further tab of indentation)

PS I forgot, of course, the stronger and more immediate argument for an extra tab to mark notes clearly as children of list items, rather than simply successors/peers

If you add one more tab, to mark them as children, ⌘/ collapsing of the notes under an item will then work properly.

Hmm. That’s interesting. I wonder why a note needs to be “double” indented to be recognized as a child? I thought I remembered at one point, notes would “fold up” to the todo item as long as they were indented under it–it wasn’t necessary to tab twice.

At any rate, it’s not a problem to double indent–just need to get in the habit.

It arises, I think, from the report-writing context in which bullet lists tend to be supporting material nested within body text.

If you look at FoldingText and Sublime displaying the same file, you can see that FoldingText’s rather clearer display of report structure involves that assumption, and grants a free (implicit) indent to bullet items.

In FT (1st of the following two images) the first tab brings a body para down to the same level as a bullet, and the second tab pushes it down to a lower level. (The 2nd shot, of Sublime, shows where the tabs are):

FoldingText:

Sublime: