Help Constructing a Complex Search with Start Dates

I’m trying to build a complex search that finds any item:

  • Not marked as done
  • Not marked as starting in the future
  • Not with a parent starting in the future

Currently the following search does #1 and #2, but I can’t figure out where to add the ancestor-or-self::* bit.

(//*) except (//@done or @start > "YYYY-MM-DD")

Thanks,
Jack

Excluding completion and future starts may be a little more efficient with a boolean (not) clause:

//not (@done or (@start > "2014-09-30"))

and the inheritors of a future start could be harvested with:

//@start > "2014-09-30"/descendant-or-self::*

at which point except will be helpful:

//not (@done or (@start > "2014-09-30")) except //@start > "2014-09-30"/descendant-or-self::*

( This assumes, of course that by Not with a parent starting in the future, you mean not with any ancestor starting in the future )

At the risk of over-elaborating, here is one way of thinking about assembling these filters:

Two steps

  1. Gathering one or more harvests, and
  2. using the set operators to ask for the overlap or combination of multiple harvests, or to subtract one harvest from another

Gathering a harvest by alternating outline steps with filters

Gathering each harvest involves a longer or shorter series of alternations:

Outline axis then filter,
then outline axis then filter,
etc …

Each harvest begins with an outline axis, even if we don’t specify one.

Often we will start with // (= anywhere in the whole outline),
and if we don’t specify a starting outline axis, the default will automatically be / (= the level 1 [top level] lines, for example single # headers)

After the outline axis, we can add a boolean filter, using brackets if it helps:

//(@start > "2014-09-30" or @done)

And after the filter, we can further refine or extend the harvest, from where we have got to so far, with another outline axis step, and a further filter.

//(not @done)/descendant-or-self::*"

The filter in this example is just a * (= anything) but it could be a more restrictive filter:

//(not @done)/descendant-or-self::(@start < "2015-09-30")

and we can go on adding alternating outline axis steps and filter steps until we have exactly the harvest that we want,

and then finally use set operators to get the combination or difference/overlap of this harvest with any other harvests which we have assembled.

@complexpoint This is immensely helpful.

You are correct that I meant “any ancestor,” not just a parent. That search works just the way I need it to—thanks!

And I appreciate the extra elaboration. I’ve been going over and over your replies for quite a bit here trying to understand why all the things I tried before weren’t working. I think I was mixing the outline axis syntax with the filter syntax.

Plus the logic with the filters is tricky. What looks like a linear hierarchical path can actually be a path doubling back on itself. But I understand somewhat it better now.

Thanks so much for your commitment to the FT community. I would be a very unhappy OmniFocus user if it weren’t for you and Jesse!

Good – I’m glad that worked – and thanks for your kind comment. Jesse has produced something really extraordinary here, and I find it a privilege to work with. Other people’s thoughts and requirements often provide a really useful prompt to making something which I find useful myself.

(Incidentally, I just tracked down that shadowing of ⌘⇧U which you noticed to the reminder tools.ftplugin (which doesn’t load if we remove smalltime, hence the puzzling sense that smalltime was the location of it).

I’ve updated the plugin at https://github.com/RobTrew/txtquery-tools, and I think the issue should be fixed if you install it and restart FT).

Rob

@complexpoint I’ve continued to use this, and it’s extremely helpful.

Today I tried to improve it (I won’t say exactly how, because I want to get there on my own!) and am getting hung up on the set operations.

I’m working with this query right now:

//not (@done or (@later/descendant-or-self::*))

Which I would think would hide:

  • All nodes marked @done, and
  • All nodes marked or having a descendant marked @later

However, in practice it hides all nodes. Is this a syntax error?

First, a quick target check

- All nodes marked or having a descendant marked @later

do you mean @later nodes and their descendants, (//@later/descendant-or-self::*)

or @later nodes and their ancestors (ancestors ‘having a descendant marked @later’ => (//@later/ancestor-or-self::*))

?

(The way you put it informally seems, at first reading, slightly different from what your path suggests)

Assuming that you want to exclude @later lines and their subtrees, you could use:

//not @done except //@later/descendant-or-self::*

This is two searches, with the harvest of one deducted from the harvest of the other

A couple of things about your first draft:

  1. We have to make a clear separation between the axis/outline steps like /descendant-or-self::, and the filter steps like not (@done or @later)

Each axis/outline step applies to the whole result of each filter step, and each filter stage acts on the whole outline/axis harvest.

If you want slightly different axis/outline patterns for @done and @later, then you need to harvest two different sets, and combine them, get their overlap, or deduct one from the other with one of the set operators (union, intersect, except).

  1. In purely syntactic terms, @later/descendant-or-self::* on its own would look a bit orphaned in the sense that although it is followed by an explicit axis step, it isn’t preceded by one, and that means that it is quietly assumed by default that it is preceded by a single ‘/’, in other words, intended only to filter top level nodes (immediate children of the root).

(Sorry I didn’t have time to write something shorter – is that more or less what you’re after ?)

Rob

or to hide the whole subtrees of both @done and @later:

//* except //(@done or @later)/descendant-or-self::*

Again two searches:

  1. Everything, except
  2. anything marked @done or @later, or descending thence

(FWIW I’ve added some basic nodepath structure diagrams at http://support.foldingtext.com/t/basic-structure-of-complex-searches/645 )

Sorry for the slow response, Rob. Yes, this is incredibly helpful, and you understood what I needed. The diagrams are also great—the “axis-filter” pattern is helpful to see in diagram form. This is coming together for me. So grateful for your explanation.

Can you help me understand the difference between not … or and not … except?

or is part of the filter conditions language (think mail rules)
[the set is defined as anything which meets this condition or meets that condition]

except is part of the set overlap language (think venn diagrams)

(different sections of those diagrams – the filters define a set, the set operations join sets, give their overlap, or deduct one set from the other)
[any member of this set except things which are also members of that other set]